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This  study  sought  to  examine  frame  of  reference  for  electronic  map  displays  and  determine  its 
effect  on  pilot  local  guidance  ability  and  global  situation  awareness  (GSA)  during  low  altitude 
rotorcraft  flights.  It  was  hypothesized  that  the  egocentric  viewpoint  would  support  superior  local 
guidance  ability,  the  exocentric  viewpoint  would  facilitate  better  GSA,  and  the  Situation 
Awareness  Rating  Technique  (SART)  would  not  be  a  good  measure  of  GSA.  Eighteen  pilots 
flew  simulated  missions  on  six  possible  paths,  with  three  paths  flown  at  200  feet  and  three  paths 
flown  at  1000  feet.  All  participants  flew  both  a  low  altitude  and  a  high  altitude  scenario  with 
each  of  three  display  viewpoints  (egocentric,  exocentric,  2D),  for  a  total  of  six  missions.  Results 
revealed  that  the  forward  field  of  view  present  in  the  simulation  appears  to  mute  the  effect  of 
display  type  on  local  guidance  ability  such  that  all  three  of  the  viewpoints  evaluated  provided 
equivalent  navigation  performance.  The  results  for  the  level  of  GSA  achieved  revealed  that  the 
egocentric  viewpoint  suffered  with  regards  to  facilitating  GSA,  but  no  differences  were  revealed 
between  the  exocentric  and  2D  viewpoints.  Furthermore,  the  GSA  results  suggested  that  the 
SART  may  not  be  a  good  measure  of  global  situation  awareness.  Results  are  discussed  in  terms 
of  map  display  design  for  rotorcraft  cockpits. 
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INTRODUCTION 

Leonardo  da  Vinci's  design  of  the  helical  air  screw,  in  1480,  is  commonly  regarded  as  the  first 
concrete  conceptualization  of  a  helicopter.  However,  actual  vertical  flight  did  not  occur  until 
Nov  13,  1907,  when  Frenchman  Paul  Cornu  made  the  first  free  flight  entirely  without  assistance 
from  the  ground,  during  which  he  achieved  a  height  of  one  foot  for  twenty  seconds.  From  this 
point  in  time,  the  helicopter  has  evolved  into  one  of  the  aviation  community's  most  versatile 
platforms,  in  that  it  can  perform  a  diverse  range  of  missions  in  a  variety  of  different 
environments,  often  while  flying  at  very  low  altitudes.  Although  this  versatility  has  contributed 
to  the  helicopter's  utility,  it  has  also  resulted  in  rotorcrafts  being  exposed  to  a  number  of  unique 
hazards  and  situations  during  flight  that  impose  significant  visual,  perceptual,  temporal,  and 
motor  demands  on  the  human  pilot  (Hart,  1988;  Padfield,  1992).  Recent  research  conducted  in 
the  fixed  wing  aircraft  domain  has  shown  that  advances  in  glass  flight  deck  display  technology 
may  help  alleviate  pilot  visual  and  perceptual  demands,  by  allowing  the  presentation  of 
navigational  information  in  a  three-dimensional  (3D)  electronic  map  display  format  (Olmos, 
Liang,  &  Wickens,  1997;  Olmos,  Wickens,  &  Chudy,  1998;  Wickens  &  Prevett,  1995;  Wickens, 
Liang,  Prevett,  &  Olmos,  1996).  However,  very  few  empirical  studies  have  been  conducted  that 
evaluate  whether  these  advantages  are  also  present  when  3D  electronic  map  displays  are  used  in 
the  context  of  a  highly  dynamic,  low  altitude  rotorcraft  environment. 

The  helicopter  is  unique  in  that  its  versatility  makes  it  capable  of  performing  missions  that 
often  dictate  that  the  aircraft  be  flown  at  low  altitudes,  which  requires  special  perception  and 
control  requirements.  Flach  and  Warren  (1995)  described  some  of  these  requirements  by  stating 
that  low  level  flight  requires  that  a  pilot  have  precision  and  responsiveness  of  control  almost  to 
the  level  of  prescience.  They  stated  that  precision  is  demanded  because  a  small  deviation  in 


2 


altitude  of  only  a  few  tens  of  feet  may  translate  into  clipping  a  tree  or  other  hazard.  Likewise, 
responsiveness  is  demanded  because  the  speed  and  distance  at  which  the  flight  occurs  results  in 
an  extremely  fast  "closure  rate",  which  leaves  a  pilot  little  time  to  act  to  achieve  mission 
objectives  or  to  react  to  avoid  a  collision.  This  means  that  the  fast  changing  spatiotemporal 
relationship  between  a  rotorcrafi  pilot  and  the  low  altitude  environment  forms  a  highly  dynamic 
and  unforgiving  ecology,  where  there  is  an  ever-present  danger  to  each  moment  of  action  or 
inaction. 

This  ever-present  danger  arises  from  the  fact  that  low  altitude  flight  places  a  pilot  in 
continuous  proximity  to  terrain  and  hazards  placed  upon  the  earth  (Hart,  1988;  Padfield,  1992), 
which  is  not  an  environment  conducive  to  safe  flight.  This  low  altitude  flight  environment  was 
poignantly  described  in  the  following  statement: 

In  the  case  of  the  rotary  wing  aircraft,  the  world  is  indeed  a  horrible  and  frightening 
place-it's  full  of  obstacles-the  earth  is  one,  but  on  top  of  the  earth  are  literally 
thousands  of  obstacles  [the  pilot]  has  to  wind  his  way  through.  And  [the  pilot]  told 
us  that  he  wants  to  have  information  about  these  obstacles. . ..In  relation  to  any 
obstacle  on  the  earth. . .  [the  pilot]  has  to  know  how  for  it  is  to  an  obstacle;  he  wants 
to  know  the  direction  it  is;  and  he  wants  to  know  the  rate  of  change  of  these  two 
things-distance  and  direction.  (Courtney,  1956,  p.51). 

If  the  rotorcrafi  flying  environment  was  a  frightening  place  in  1956,  then  it  has  become  truly 
terrifying  in  recent  years,  as  more  and  more  powerlines  and  telecommunications  towers  and 
antennas  have  been  constructed.  Pilots  must  be  aware  of  these  man-made  hazards  and  naturally 
occurring  terrain  and  weather  hazards,  while  at  the  same  time  trying  to  navigate  an  inherently 
unstable  aircraft  to  a  desired  location  in  order  to  meet  the  mission  requirements. 
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Unfortunately,  research  has  shown  that  rotorcraft  pilots  sometimes  have  very  little  time  to  do 
anything  beyond  the  actual  navigation  of  the  aircraft.  An  empirically  validated  task  analysis  of 
nap-of-the-earth  and  low-level  helicopter  operations  found  that  helicopter  pilots  devote 
approximately  70%  of  their  time  to  navigational  activities  when  flying  in  an  unfamiliar 
environment  (Shaffer,  Hendy,  &  White,  1988).  This  means  that  there  is  little  time  and  few 
resources  available  for  the  pilots  to  obtain  and  maintain  sufficient  awareness  of  their 
surroundings,  or  what  many  term  achieving  situational  awareness  (SA).  Because  of  the 
numerous  hazards  present  within  the  helicopter's  low  altitude  flying  environment,  a  low  level  of 
pilot  situational  awareness  can  have  disastrous  consequences.  Therefore,  the  goal  of  the  research 
presented  here  is  to  evaluate  how  different  frames  of  reference  for  electronic  map  displays  affect 
a  pilot's  local  guidance  ability  and  situation  awareness  during  low  altitude  rotorcraft  missions. 

Since  our  evaluation  of  the  effects  of  different  electronic  map  frames  of  reference  will  be 
based  on  the  pilot's  ability  to  maintain  local  guidance  and  achieve  situation  awareness,  it  seems 
appropriate  to  discuss  these  two  tasks  as  they  apply  to  the  rotorcraft  domain.  Therefore,  we  will 
first  define  each  of  these  tasks,  discuss  why  they  are  important,  and  describe  ways  in  which 
performance  on  each  task  can  be  measured.  We  will  then  proceed  with  a  discussion  of  electronic 
map  frames  of  reference  and  present  findings  from  previous  research  in  this  area,  highlighting 
those  studies  that  measured  local  guidance  ability  and/or  situation  awareness.  Finally,  we  will 
conclude  the  introduction  with  a  thumbnail  sketch  of  previous  research  findings,  explain  our 
independent  variables,  and  present  our  hypotheses  for  this  study. 
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Assessment  methods 

In  the  rotorcraft  aviation  domain,  local  guidance  and  situation  awareness  are  not  independent 
tasks.  This  is  because  the  two  tasks  are  often  performed  concurrently  during  flight  (Olmos  et  al., 
1997;  Wickens  &  Prevett,  1995).  However,  in  order  to  understand  what  each  is  and  why  they  are 
important,  we  will  discuss  them  separately  in  this  section. 

Local  guidance 

Local  guidance  involves  maintaining  the  aircraft  on  the  desired  flight  path  through  the  three- 
dimensional  airspace  (Olmos  et  al.,  1998;  Wickens  &  Prevett,  1995).  This  task  is  considered  a 
form  of  closed- loop  tracking  that  requires  knowledge  of  one's  momentary  position  and 
orientation  in  relationship  to  the  forward  field  of  view  (FFOV)  seen  out  of  the  cockpit  (Olmos  et 
al,  1997;  Wickens  &  Prevett,  1995).  The  key  to  local  guidance  is  to  have  a  momentary 
representation  of  the  error  from  the  ideal  flight  path,  along  with  an  understanding  of  the  required 
actions  necessary  to  close  the  control  loop  and  correct  that  error  (Olmos  et  al.,  1997;  Wickens  & 
Prevett,  1995).  Because  of  the  demanding  nature  of  low  altitude  helicopter  flight,  this  can  be  a 
nontrivial  task. 

While  it  is  relatively  easy  to  define  the  construct  of  local  guidance,  being  able  to  determine  its 
relative  importance  in  terms  of  vertical  and  lateral  error  is  more  difficult.  The  question  is  how 
important  is  50  feet  of  vertical  deviation  versus  50  feet  of  lateral  deviation?  It  intuitively  makes 
sense  that  vertical  errors  are  more  important,  since  there  is  less  vertical  space  in  which  to 
maneuver  as  compared  to  lateral  space.  However,  fifty  feet  of  vertical  deviation  while  flying  at 
an  altitude  of  5000  feet  is  not  nearly  as  critical  as  the  same  deviation  while  flying  at  100  feet. 

The  same  is  true  for  lateral  deviations.  At  higher  altitudes,  lateral  deviations  from  the  flight  path 
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may  result  in  the  pilot  merely  becoming  lost  within  the  airspace.  These  same  lateral  deviations 
during  low  level  flight  can  result  in  the  pilot  colliding  with  the  ground  (if  the  aircraft  is  flying  in 
mountainous  terrain)  or  with  hazards  placed  upon  the  ground.  Therefore,  the  importance  for 
precise  local  guidance  is  tied  to  the  altitude  of  flight  at  which  an  aircraft  is  operating,  because 
vertical  and  lateral  deviations  at  low  altitude  flight  levels  can  have  drastically  greater 
consequences  than  the  same  deviations  at  higher  altitudes.  Since  many  rotorcraft  missions 
require  the  pilot  to  fly  at  low  altitudes,  where  they  are  always  in  close  proximity  to  terrain  and 
man-made  hazards,  the  need  for  precise  local  guidance  during  these  flights  is  of  paramount 
importance. 

Measuring  a  pilot's  local  guidance  ability  is  fairly  straightforward  and  easy  in  a  simulation 
environment.  It  involves  measuring  the  difference  between  the  point  where  the  aircraft  is  at  a 
given  time  and  the  point  where  the  aircraft  should  be  (the  ideal  programmed  flight  path)  at  that 
time.  Computer  technology  allows  researchers  to  measure  these  deviations  throughout  the 
duration  of  a  simulated  mission. 

Situation  awareness 

Situation  awareness  is  a  ubiquitous  term  used  in  the  aviation  community  and  is  intuitively 
understood  to  be  a  good  thing  for  a  pilot  to  have.  A  formal  definition  for  this  construct  was 
somewhat  hard  to  develop,  because  S  A  is  a  mental  process  that  must  be  inferred  from  behavioral 
evidence,  which  is  not  a  trivial  matter,  and  therefore  the  construct  was  not  very  well  understood. 
However,  Endsley  (1988)  formulated  a  widely  accepted  definition  which  states  that  situation 
awareness  is  "the  perception  of  the  elements  in  the  environment  within  a  volume  of  time  and 
space,  the  comprehension  of  their  meaning,  and  the  projection  of  their  status  in  the  near  future  . 


6 


This  definition  contains  three  distinct  levels  of  SA,  which  she  defined  as  perceiving  critical 
factors  in  the  environment  (Level  1  SA),  understanding  what  those  factors  mean,  particularly 
when  integrated  together  in  relation  to  the  aircrew's  goals  (Level  2),  and  an  understanding  of 
what  will  happen  with  the  system  in  the  near  future  (i.e.,  prediction)  (Level  3)  (Endsley,  1998). 

Within  this  broad,  general  concept  of  situation  awareness  are  certain  subsets,  or  different 
elements,  of  awareness  that  an  individual  must  obtain  in  order  to  achieve  SA  (Endsley,  1998). 
Recent  research  by  Entin  (1998)  suggests  that  in  a  dynamic  and  fluid  situation,  such  as  an  attack 
helicopter  mission,  overall  detailed  SA  may  actually  fragment  into  a  number  of  relatively 
independent  components.  Therefore,  where  possible,  one  should  refine  the  definition  more 
specifically  to  refer  to  more  specific  constructs,  such  as;  geographical  hazard  awareness, 
automation  mode  awareness,  or  task  awareness  (Wickens,  in  press).  One  of  the  elements  which 
is  critical  for  low  altitude  rotorcraft  missions  is  spatial,  or  hazard  awareness  (Endsley,  1998). 
Hazard  awareness  involves  being  aware  of  any  hazard  (e.g.,  terrain,  weather,  other  traffic,  no-fly 
zones)  within  some  proximity  of  the  aircraft  (Olmos  et  al.,  1998;  Wickens,  Fadden,  Merwin,  & 
Ververs,  1998),  which  is  extremely  important  because  of  the  numerous  hazards  present  in  the 
low  altitude  rotorcraft  environment.  While  a  pilot  must  be  aware  of  the  hazards  near  the  forward 
flight  path,  he  or  she  must  also  have  an  awareness  of  the  hazards  within  a  360  degree  sphere 
around  the  aircraft,  or  what  is  termed  global  situation  awareness  (GSA)  (Olmos  et  al.,  1997; 
Wickens  et  al.,  1998;  Wickens  &  Prevett,  1995).  The  need  for  precise  global  situation  awareness 
by  rotorcraft  pilots  is  necessitated  by  the  helicopter's  ability  to  move  backwards  while  hovering, 
which  means  that  the  pilot  must  know  whether  there  are  obstacles  or  hazards  behind  the  aircraft. 

The  true  importance  of  SA  becomes  apparent  by  discussing  the  role  that  errors  in  situation 
awareness  play  in  aircraft  incidents  and  accidents.  We  will  first  look  at  general  findings  in  the 
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aviation  industry,  and  then  present  statistics  specific  to  the  rotorcrafi  domain.  Jones  and  Endsley 
(1996)  conducted  an  evaluation  of  143  Aviation  Safety  Reporting  System  (ASRS)  reports 
collected  between  January  1986-May  1992  that  included  SA  errors  by  either  pilots  or  air  traffic 
control  personnel.  They  found  that  76.3%  of  the  situation  awareness  errors  were  Level  1  SA, 
which  involved  a  failure  to  perceive  information  or  the  misperception  of  information,  20.3% 
were  Level  2  errors,  which  involved  the  improper  integration  or  comprehension  of  information 
that  is  perceived,  and  3.4%  were  Level  3,  which  involved  cases  in  which  a  projection  of  future 
actions  of  the  system  was  incorrect  or  missing.  The  percentage  of  errors  for  each  level  of  SA 
found  in  the  incident  reports  were  very  similar  to  an  earlier  study  by  Endsley  (1995),  who  looked 
at  situation  awareness  errors  involved  in  24  major  air  earner  accidents  from  1989  to  1992.  Of 
the  32  actual  SA  errors  identified  within  these  major  accidents,  72%  were  Level  1  errors,  22% 
were  Level  2,  and  6%  were  level  3. 

Accident  statistics  specifically  looking  at  the  number  of  helicopter  accidents  also  highlight  the 
importance  of  SA  for  rotorcraft  pilots.  The  Flight  Safety  Foundation  performed  an  analysis  of 
information  provided  from  the  Federal  Aviation  Association's  Accident/Incident  Database 
System  of  turbine  helicopters  flown  in  the  United  States  during  1988-1992.  During  this  period, 
85  fatal  turbine  helicopter  accidents  occurred  resulting  in  172  fatalities,  with  62  of  the  accidents 
being  attributed  to  pilot  error  (Harris,  1995).  These  62  accidents  were  then  classified  according 
to  a  taxonomy  developed  by  the  National  Transportation  Safety  Board,  which  includes  six 
accident  categories:  weather-related,  mechanical  failure,  obstacle  strikes,  control  loss,  midair 
collision,  and  miscellaneous.  Placing  the  accidents  in  these  categories  revealed  that  38  out  of  the 
62  pilot  error  accidents  were  either  weather  related  or  a  result  of  terrain,  obstacle,  or  midair 


collisions. 
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Although  this  study  did  not  specifically  classify  these  accidents  according  to  SA  errors,  one 
can  infer  that  situation  awareness  played  a  role  in  these  accidents,  since  the  pilots  would  not  have 
entered  or  collided  with  the  hazards  if  they  had  been  aware  of  them.  Also,  if  the  SA  errors 
present  in  these  accidents  were  classified  according  to  the  level  of  SA  during  which  they 
occurred,  the  percentages  found  would  probably  be  very  similar  to  Endsley's  (1996,  1995) 
studies.  In  both  of  these  studies.  Level  1  SA  errors  were  the  most  common  errors  and  were 
defined  as  failures  to  perceive  information  or  the  misperception  of  information.  This  also  seems 
to  be  the  case  for  these  38  fatal  helicopter  crashes,  again  because  the  pilots  would  not  have 
entered  or  collided  with  the  hazards  had  they  correctly  perceived  them. 

As  stated  before,  there  are  certain  subcomponents,  or  elements,  of  awareness  that  a  person 
must  obtain  in  order  to  achieve  SA  (Endsley,  1988).  These  subcomponents  are  interactive  in 
nature,  such  that  improving  situation  awareness  for  one  subcomponent  may  actually  reduce  SA 
on  another  (Endsley,  1 996).  Because  of  this,  as  well  as  the  overall  hypothetical  nature  of  S  A, 
multiple  measurement  techniques  need  to  be  employed  to  try  and  folly  measure  the  many 
different  facets  of  SA  present  during  a  particular  mission.  Relying  solely  on  one  measure  for 
situation  awareness  can  yield  misleading  results,  and  therefore,  each  measure  must  be  viewed  in 
the  context  of  other  measures  of  SA  (Endsley,  1996).  Endsley  (1996)  provided  a  comprehensive 
review  of  the  different  approaches  and  measurement  techniques  that  may  be  employed  to 
measure  S  A.  Based  on  her  review,  we  will  use  the  remaining  part  of  this  section  to  briefly 
describe  several  of  these  approaches  and  techniques. 

One  possible  approach,  although  it  may  only  give  an  indirect  measure  of  situation  awareness, 
is  to  examine  the  processes  used  for  acquiring  SA  (Adams,  Tenney,  &  Pew,  1995).  This  may 
provide  information  concerning  how  an  operator  allocates  his  or  her  attention  in  using  a 
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particular  system  design,  which  could  reveal  the  relative  priority  of  different  types  of  information 
and  the  relative  utility  of  information  sources.  Several  different  measures  may  be  employed, 
such  as  eye  tracking,  to  assess  how  attention  is  deployed  in  the  process  of  acquiring  SA,  verbal 
protocols,  to  gain  some  insight  into  how  the  obtained  information  is  integrated  and  used  in  the 
process,  and  evaluation  of  verbal  communications,  to  determine  the  types  of  information  that  is 
lacking  from  displays  and  verbal  techniques  used  for  acquiring  SA.  Although  these  measures  do 
not  provide  any  insight  into  how  the  attended  to  information  is  used  or  combined  to  form  higher 
level  SA,  when  used  in  conjunction,  they  may  offer  valuable  information  concerning  the 
processes  an  individual  uses  to  obtain  SA.  This  information  may  help  in  the  development  of 
training  strategies  designed  to  help  individuals  take  certain  steps  which  may  help  in  achieving 
overall  situation  awareness. 

Other  measures  have  been  developed  that  assess  the  product  of  situation  awareness  directly, 
and  are  usually  broken  down  into  three  categories:  subjective  techniques,  questionnaires,  and 
performance  measures.  Subjective  techniques  usually  involve  the  operator  making  a  subjective 
estimation  of  how  much  SA  he  or  she  has  when  using  a  particular  system  design.  One  of  the  best 
known  subjective  scales  for  measuring  SA  is  the  Situational  Awareness  Rating  Technique 
(SART)  developed  by  Taylor  (1990).  The  SART  allows  operators  to  rate  a  system  or  design 
based  on  the  amount  of  demand  on  attentional  resources,  supply  of  attentional  resources,  and 
understanding  of  the  situation  provided  (Taylor  1990).  Follow  on  studies  of  the  SART's 
effectiveness  and  usefulness  found  that  the  rating  scale  is  correlated  with  performance  measures, 
and  when  used  with  performance  measures  may  provide  a  powerful  tool  for  system  design 
(Selcon  &  Taylor,  1990).  This  approach  is  attractive  because  it  is  fairly  inexpensive  and  is  easy 
to  administer,  however,  it  may  not  provide  an  accurate  quantification  of  situation  awareness. 
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This  is  because  the  operators  may  not  know  of  their  own  inaccuracies  or  the  information  they  are 
unaware  of,  and  therefore  they  have  a  limited  basis  for  making  a  judgement  of  SA.  Another 
consideration  is  that  the  demand  on  attentional  resources  and  supply  of  attentional  resources 
categories  are  more  closely  associated  with  the  measure  of  workload.  As  such,  a  person  s  self- 
ratings  may  be  highly  influenced  by  self-assessments  of  performance  under  different  workload 
conditions,  which  may  cause  the  ratings  to  be  biased  by  issues  outside  of  the  SA  construct.  It  is 
for  this  reason  that  in  the  current  experiment  we  will  evaluate  all  three  of  the  SART  categories, 
but  we  will  focus  on  the  understanding  category  as  the  most  valid  of  the  three  for  measuring  SA. 

The  second  direct  measure  technique,  questionnaires,  provides  an  objective  assessment  of  SA 
by  obtaining  detailed  information  about  subjective  perceptions  and  then  comparing  them  against 
reality.  A  critical  consideration  when  using  this  technique  is  deciding  when  the  information 
should  be  gathered.  Administering  the  questionnaire  posttest  may  present  some  problems, 
because  memories  of  dynamic  situation  awareness  will  be  less  reliable  with  time.  Therefore, 
posttest  administration  may  reliably  capture  only  the  operator  SA  at  the  very  end  of  a  trial. 
Presenting  the  questionnaire  while  the  operator  is  performing  the  simulated  task  overcomes  this 
deficiency,  however,  it  is  intrusive  in  that  it  may  alter  SA  and  task  performance.  This  may  occur 
because  responding  to  the  queries  during  task  performance  provides  an  additional  secondary 
task.  Another  possibility  is  to  freeze  the  entire  simulation  at  randomly  selected  times,  blank  the 
screens,  and  then  query  the  operators  as  to  their  perceptions  of  the  situation  at  the  time.  As  an 
example,  Endsley  (1987)  developed  the  Situation  Awareness  Global  Assessment  Technique 
(SAGAT),  which  is  a  tool  that  uses  temporary  freezes  of  the  simulation,  during  which  the 
subjects  are  queried  about  all  SA  requirements,  including  Level  1,  Level  2,  and  Level  3  situation 
awareness.  This  approach  may  limit  possible  biases  of  attention,  because  the  subjects  cannot 


prepare  for  the  queries  in  advance,  since  they  could  be  queried  over  almost  any  aspect  of  the 
situation.  Considerable  care  must  be  taken,  however,  in  the  development  of  the  battery  of 
queries  to  be  administered,  in  order  to  ensure  that  all  of  the  SA  requirements  (Level  1,2,3)  are 
included.  There  is  also  an  associated  disadvantage  with  this  technique,  which  involves  the 
temporary  halt  of  the  simulation. 

Because  the  SART  (Taylor,  1990)  and  SAG  AT  (Endsley,  1987)  are  two  of  the  most 
commonly  used  SA  measures,  Endsley,  Selcon,  Hardiman  and  Croft  (1998)  sought  to  provide  a 
direct  comparison  of  the  two  measures  when  used  within  a  display  evaluation  study.  The 
researchers  utilized  two  different  navigation  displays,  with  one  display  configuration  providing 
visible  threat  envelopes  revealing  information  concerning  pop-up  threats  and  the  other 
configuration  not  presenting  the  threat  envelopes.  In  the  study,  pilots  were  required  to  navigate 
to  different  waypoints  during  a  low  level  flight  scenario.  Pop-up  threats  appeared  during  the 
flight  which  pilots  were  required  to  avoid,  while  at  the  same  time  staying  as  close  to  their 
predetermined  flight  path  as  possible.  Pilot  performance  was  measured  in  terms  of  RMS  error 
from  the  assigned  course  and  the  total  time  spent  inside  the  threats'  launch  zone,  and  SAGAT 
and  SART  were  administered  during  freezes  which  occurred  at  random  times  during  the  threat 
avoidance  task. 

Endsley  et  al.  (1998)  found  that  both  of  the  measures  contributed  sensitivity  and  diagnosticity 
regarding  the  effects  of  the  display  concept.  The  overall  SART  rating  was  found  to  be 
significantly  higher  with  the  threat  envelopes  than  without,  a  difference  which  was  mainly 
attributable  to  differences  in  subject  ratings  of  the  underlying  dimension  of  understanding.  On 
the  other  hand,  results  for  the  SAGAT  measures  revealed  that  while  the  threat  envelopes 
enhanced  the  pilots  S  A  regarding  their  own  heading  and  roll  attitude,  the  envelopes  actually 
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decreased  the  pilots  SA  regarding  the  location  of  threat  aircraft  and  their  ability  to  determine 
whether  the  pop-up  would  be  a  threat  if  they  maintained  their  current  heading.  The  researchers 
conducted  a  correlation  analysis,  which  revealed  that  there  was  no  correlation  between  the  SART 
components  (demand  on  attention,  supply  of  attention,  and  understanding)  and  the  SAG  AT 
measures.  The  researchers  cautioned  that  since  the  subjective  SART  and  objective  SAGAT 
measures  were  not  correlated,  some  doubt  arises  as  to  the  validity  of  the  SART  as  an  indication 
of  a  person's  actual  SA.  However,  the  SART  was  highly  correlated  with  confidence  level  in  SA 
and  subjective  rating  of  performance,  which  provides  the  level  of  a  person's  perceived  quality  of 
SA.  This  may  be  important  in  determining  how  a  person  will  choose  to  act  on  their  SA, 
independent  of  the  actual  quality  of  their  SA. 

The  final  direct  technique  or  approach  we  will  describe  for  measurement  of  SA  is  the  use  of 
performance  measures.  This  approach  is  advantageous  in  that  these  measures  are  objective  and 
usually  nonintrusive,  since  simulation  computers  can  be  programmed  to  record  specific 
performance  data  automatically.  Researchers  generally  agree  that  global  performance  measures 
are  not  useful  for  measurement  of  SA,  because  assessing  the  end  result  of  a  task  reveals  little 
about  the  situation  awareness  that  occurred  throughout  the  task.  A  way  to  avoid  this  problem  is 
to  embed  specific  tasks  within  the  context  of  an  ongoing  scenario  that  tap  the  implicit  knowledge 
of  the  different  dimensions  of  SA.  An  example  of  this  would  be  to  incorporate  a  disorienting 
event  that  often  leads  to  a  loss  of  S  A  and  assess  an  individual's  ability  to  determine  the  present 
attitude  and  the  desired  future  attitude  following  the  disorienting  event.  From  such  events,  an 
individual's  level  of  SA  can  be  inferred  from  the  nature  and  timing  of  choices  made  in  response 
to  the  event.  Wickens  and  Haskell  (1993)  incorporated  this  technique  into  a  study  evaluating  a 
3D  and  a  2D  navigation  display  used  during  microwave  landing  systems  approaches.  During  the 
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approaches,  the  pilots  had  to  fly  through  several  disorienting  events,  including  5-second  wind 
shears,  5-second  display  failures,  and  combined  events  consisting  of  a  wind  shear  followed  by  a 
display  failure  after  2  seconds.  Pilots  were  told  which  type  of  disturbance  to  expect  at  the 
beginning  of  the  experiment,  however,  they  were  not  given  any  information  concerning  when 
and  where  on  each  path  the  disturbance  events  would  occur.  Wickens  and  Haskell  measured 
each  pilot's  flight  control  accuracy  before,  during  and  after  the  disturbance  events,  from  which 
they  were  able  to  infer  which  display  facilitated  quicker  and  more  accurate  recovery.  Their 
results  suggested  that  although  the  3D  display  provided  superior  flight-path  tracking  accuracy 
during  normal  flight,  it  might  hinder  performance  during  the  disturbance  events.  One  also  needs 
to  keep  in  mind  that,  as  with  developing  questionnaires,  considerable  preplanning  must  be 
conducted  to  ensure  a  plausible  experimental  scenario  that  mimics  realistic  aspects  of  the  real 
world. 

These  descriptions  of  the  various  techniques  reveal  that  each  has  its  advantages,  but  also  some 
related  costs.  This  serves  to  highlight  the  point  that  a  multiple  technique  approach  must  be  taken 
when  trying  to  measure  situation  awareness.  Rather  than  focusing  on  one  particular 
measurement  tool,  researchers  need  to  use  several  techniques  and  then  evaluate  the  degree  to 
which  these  multiple  measures  tend  to  agree  or  disagree  in  regards  to  how  they  are  affected  by  a 
specific  display  or  design  condition  and  its  ability  to  promote  situation  awareness. 

Thus,  having  defined  and  discussed  local  guidance  and  situation  awareness,  we  will  now 
proceed  into  a  discussion  of  frames  of  reference  (FOR)  and  their  relevance  for  electronic  map 
display  design,  because  this  is  the  critical  independent  variable  that  we  will  use  to  evaluate  pilot 
local  guidance  ability  and  the  correspondence  or  congruence  of  different  SA  measures.  We  will 
first  explain  different  frames  of  reference  and  their  characteristics.  Then  we  will  continue  with  a 
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discussion  of  the  design  parameters  that  determine  an  electronic  map  display's  frame  of 
reference.  Finally,  we  will  present  previous  research  that  has  evaluated  electronic  map  displays 
using  different  frames  of  reference,  highlighting  those  studies  that  measured  local  guidance 
and/or  global  situation  awareness  in  an  aviation  context. 

Frame  of  reference 

As  discussed  earlier,  local  guidance  requires  that  a  pilot  have  knowledge  of  his  or  her  current 
position  with  regards  to  the  specified  ideal  flight  path,  or  in  simpler  terms,  he  or  she  needs  to 
know  "where  am  I"  at  any  given  moment.  This  knowledge  requires  a  highly  ego-centered  frame 
of  reference  (ERF),  which  is  characterized  as  a  small  scale,  forward  looking  perspective  (3D) 
view  of  the  world  (Wickens,  1998).  The  ERF  is  typified  by  what  pilots  see  in  their  FFOV  and  is 
defined  in  terms  of  left,  right,  above,  below,  and  near-far  (Wickens,  1998).  On  the  other  hand, 
global  situation  awareness  requires  that  a  pilot  have  knowledge  of  potential  hazards  that  are  not 
visible  in  the  FFOV.  This  involves  pilots  knowing  where  these  hazards  are  in  a  more  world- 
centered  frame  of  reference  (WRF),  which  means  that  the  location  of  the  hazards  is  always  the 
same  independent  of  the  location  and  movement  of  the  aircraft  (Wickens,  1998). 

This  discussion  shows  that  these  two  tasks  require  very  different  knowledge  states  in  order  to 
perform  them  successfully.  A  simple  solution  for  this,  if  they  were  independent,  discrete  tasks, 
would  be  to  develop  two  separate  displays  to  support  each  frame  of  reference.  Unfortunately, 
the  two  tasks  are  often  performed  concurrently  during  flight,  which  means  that  a  pilot  would 
have  to  switch  back  and  forth  between  the  two  displays.  This  would  lead  to  a  high  level  of 
mental  workload,  resulting  from  the  mental  transformations  required  to  cognitively  integrate  the 
frame  of  reference  information  across  the  two  displays.  The  increased  mental  workload  caused 


by  these  mental  gymnastics  suggests  that  there  needs  to  be  some  form  of  display  integration,  in 
order  to  support  both  the  ERF  and  WRF  within  one  display. 
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Researchers  have  realized  that  3D  electronic  map  displays  allow  for  the  presentation  of  an 
integrated  view  of  these  different  frames  of  reference  (Olmos,  Liang,  &  Wickens,  1997;  Olmos, 
Wickens,  &  Chudy,  1998;  Wickens  &  Prevett,  1995;  Wickens,  Liang,  Prevett,  &  Olmos,  1996). 
However,  depending  on  a  number  of  design  parameters,  this  presentation  can  vary  dramatically 
as  to  the  level  of  ERF  and  WRF  represented.  Therefore,  we  will  now  transition  into  a  discussion 
of  the  parameters  that  determine  a  3D  electronic  map  display's  frame  of  reference. 

Electronic  maps 

In  the  design  of  an  electronic  map,  frame  of  reference  is  defined  by  the  viewpoint,  relative  to 
the  momentary  FFOV,  that  is  presented  to  the  pilot  (Wickens,  1998).  As  such,  the  actual  frame 
of  reference  adopted  by  a  display  is  a  function  of  three  parameters:  the  location  of  the  viewpoint, 
the  relative  motion  of  the  viewpoint,  and  the  attitude  of  the  viewpoint.  The  location  of  the 
viewpoint  (represented  by  cameras)  determines  the  field  of  view  (FOV)  that  is  presented  by  the 
electronic  map  display.  When  the  viewpoint  is  placed  in  the  aircraft,  as  shown  in  panel  A  and  B 
of  Figure  1,  an  immersed  perspective  or  egocentric  FOV  is  presented,  which  essentially  mimics 
the  pilot's  FFOV  (Wickens,  1998;  Wickens  et  al.,  1998).  In  contrast,  when  the  viewpoint 
location  is  removed  from  the  FFOV,  the  viewpoint  presents  an  exocentric  FOV,  which  shows  the 
aircraft  in  the  context  of  its  surroundings  (see  panel  C,  Figure  1)  (Wickens,  1998). 

Figure  1.1:  Four  different  frames  of  reference  for  depicting  aircraft  navigation  information. 
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The  motion  of  the  viewpoint  determines  whether  the  FOV  presented  by  the  location  of  the 
viewpoint  translates  and  rotates  with  the  movement  and  rotation  of  the  aircraft.  This  depends  on 
whether  or  not  any  single  projection  parameter,  such  as  azimuth  angle,  elevation  angle,  or 
distance  vector,  is  linked  directly  to  the  motion  of  the  aircraft.  If  any  parameter  is  heading 
slaved,  then  it  is  directly  linked  to  the  motion  of  the  aircraft,  which  means  the  presented  FOV  is 
constantly  determined  by  the  momentary  direction  of  travel  of  the  aircraft  (Wickens,  1998).  In 
other  words,  a  heading  slaved  viewpoint  results  in  a  track-up  map  that  is  always  aligned  with  the 
aircraft's  direction  of  travel  and  corresponds  to  a  pilot's  ERF.  On  the  other  hand,  if  the  viewpoint 
is  azimuth  fixed,  then  none  of  the  projection  parameters  are  linked  to  aircraft  motion  and  the 
displayed  FOV  remains  fixed.  This  results  in  a  north-up  map,  which  corresponds  to  the  pilot's 
WRF.  An  easy  way  to  envision  this  is  that  when  viewpoint  motion  is  azimuth  fixed,  heading  and 
lateral  position  changes  will  be  indicated  on  the  map  display  by  a  movement  of  the  aircraft 
symbol,  while  a  heading  slaved  map  display  will  represent  these  changes  by  a  movement  of  the 
full  display  in  the  opposite  direction  (Wickens,  1998). 

Viewpoint  attitude  is  the  final  parameter  for  determining  electronic  map  display  frame  of 
reference.  The  viewpoint  attitude,  or  its  elevation  angle,  may  be  slaved  to  the  pitch  of  the 
aircraft  (Wickens,  1998).  This  usually  occurs  when  the  viewpoint  is  in  an  exocentric  location 
where  the  viewpoint  follows  the  aircraft  as  if  attached  to  it  by  a  rigid  pole,  commonly  known  as  a 
"tethered"  view  (Wickens  &  Prevett,  1995;  Wickens,  1998).  The  viewpoint  elevation  angle  may 

also  be  fixed  at  any  value  between  0°  and  -90°,  with  the  -90°  fixed  elevation  angle  resulting  in  a 
two-dimensional  (2D)  planar  map.  Because  this  view  results  in  complete  compression  of 

graphical  information  along  the  vertical  axis,  this  type  of  view  is  often  coupled  with  a  0°  vertical 


display  in  order  to  present  foil  information  about  the  3D  airspace,  to  form  a  co-planar  display 
(see  panel  D,  Figure  1)  (Wickens,  1998). 
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Thus,  it  can  be  seen  that  the  particular  combination  of  these  design  parameters  determines 
how  the  airspace  is  displayed  to  the  pilot  (Wickens,  1998).  Manipulation  of  these  parameters 
can  affect  the  degree  to  which  the  map  display  depicts  ERF  and  WRF  information,  such  that  the 
FOR  can  be  thought  to  fall  along  a  continuum  of  egocentricity.  In  light  of  the  knowledge 
required  for  local  guidance  and  global  situation  awareness,  the  frame  of  reference  adopted  by  an 
electronic  map  display  can  have  critical  implications  concerning  the  pilot's  ability  to  perform 
these  tasks  effectively.  We  will  now  discuss  previous  studies  that  have  evaluated  the  effects  of 
different  FORs  adopted  by  electronic  map  displays  and  their  effect  on  a  pilot's  local  guidance 
ability  and/or  global  situation  awareness.  This  discussion  will  be  divided  up  into  three  sections 
according  to  the  type  of  displays  that  result  from  manipulation  of  each  of  the  parameters:  track- 
up  versus  north-up  (azimuth  motion),  egocentric  versus  exocentric  (location),  and  perspective 
versus  planar  (attitude). 

Track-up  versus  north-up  viewpoint 

Manipulation  of  the  projection  parameters  for  viewpoint  motion  determines  whether  an 
electronic  map  display  uses  a  track-up  or  a  north-up  format.  Traditionally,  most  maps  and 
navigational  charts  have  been  designed  in  a  north-up  fashion,  which  means  the  world  presented 
to  the  pilot  remains  in  a  fixed  position  representing  the  WRF,  unless  the  pilot  chooses  to  rotate  it 
by  hand  (Williams,  Hutchinson,  &  Wickens,  1996).  For  flight  in  any  direction  besides  due  north, 
this  type  of  design  requires  mental  rotation  on  the  part  of  the  pilot  in  order  to  align  the  WRF  with 
the  ERF,  or  what  is  viewed  out  of  the  cockpit.  This  cognitive  demand  can  be  eliminated  by  the 


use  of  a  rotating,  or  track-up  map,  which  constantly  rotates  and  translates  to  correspond  to  the 
aircrafts  location  and  heading. 
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The  issue  of  map  rotation  has  received  a  fair  amount  of  investigation  over  the  past  several 
years  with  regards  to  its  effects  on  navigation  and  situational  awareness.  Several  studies  have 
shown  that  rotating  maps  produce  greater  flight  path  tracking  accuracy  and  navigational 
efficiency  (Aretz,  1991;  Andre,  Wickens,  Moorman,  &  Boschelli,  1991;  Olmos  et  al.,  1997;  Rate 
&  Wickens,  1993;  Wickens  et  al.,  1996).  This  is  because  north-up  maps  impose  resource,  or 
workload,  costs  during  navigation,  which  translates  into  performance  decrements.  When  a  pilot 
flies  a  southerly  heading  using  a  fixed  map,  landmarks  displayed  to  the  left  on  the  map  are 
actually  to  the  right  in  the  view  of  the  world,  which  requires  the  pilot  to  perform  time  and  effort 
consuming  mental  rotation  to  bring  the  two  views  into  congruence  (Aretz,  1991;  Olmos  et  al., 
1997;  Rate  &  Wickens,  1993;  Wickens  et  al.,  1996).  Therefore,  track-up  maps  are  preferred 
when  flight  profiles  require  rapid  decisions  regarding  lateral  turns  (Wickens  et  al.,  1996). 

The  advantages  of  rotating  maps  have  not  been  as  clear  cut,  however,  concerning  situational 
awareness.  Some  studies  have  found  that  north-up  maps  better  support  the  learning  of  an 
environment  (Aretz,  1991;  Barfield  et  al.,  1995;  Williams,  Wickens,  &  Hutchinson,  1996).  In 
each  of  these  studies,  a  post-trial  map  reconstruction  task  was  utilized  which  required  the  subject 
to  recall  as  many  landmarks  and  flight  path  characteristics  as  possible.  Results  revealed  that  the 
north-up  map  resulted  in  better  subject  performance  for  this  task.  The  researchers  concluded  that 
this  was  because  the  landmarks  and  features  of  the  world  are  presented  in  a  constant  frame  of 
reference  (i.e.,  WRF),  which  allows  the  user  to  more  quickly  create  a  mental  map  of  the 
surroundings.  This  mental  map  represents  global  spatial  knowledge  that  results  in  increased 
situation  awareness,  particularly  global  situation  awareness. 
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On  the  other  hand,  other  studies  have  not  found  north-up  maps  to  be  more  advantageous  for 
situational  awareness.  A  study  conducted  by  Rate  and  Wickens  (1993)  evaluated  the  effects  of 
map  display  rotation  on  situational  awareness  in  landing  approaches.  Measurement  of  situation 
awareness  was  assessed  in  terms  of  error  and  response  latency  to  probed  questions  regarding 
terrain  features.  Their  results  suggested  there  were  no  significant  differences  between  the  north- 
up  and  track-up  displays  in  terms  of  situation  awareness.  Wickens  et  al.  (1996)  conducted  an 
experiment  to  extend  the  work  done  by  Rate  and  Wickens  (1993),  which  looked  at  pilot 
performance  during  terminal  area  navigation.  They  used  a  position  report  task,  a  frozen-screen 
test,  and  a  map  reconstruction  task  to  assess  the  level  of  global  situation  awareness.  Results  for 
this  study  revealed  that  the  rotating,  track-up  map  display  was  not  detrimental  to  the  situation 
awareness  tasks.  Wickens  et  al.  (1996)  concluded  that  using  a  north-up  map  while  actively 
flying  might  result  in  mental  rotation  workload  costs  that  neutralize  and  thereby  eliminate  any 
fixed-map  benefits  to  the  long-term  retention  of  environmental  structure. 

Olmos  et  al.  (1997)  utilized  the  same  paradigm  as  Wickens  et  al.  (1996),  however,  they 
incorporated  a  simulated  FFOV  to  allow  pilots  the  opportunity  for  direct  observation  of 
surrounding  terrain,  simulating  visual  meteorological  conditions  (VMC).  They  used  a  position 
report  task,  FFOV-map  comparison  task,  direction-indicating  task,  and  map  reconstruction  as 
measures  of  situation  awareness  during  navigation  of  terminal  area  approach  paths.  Overall, 
results  suggested  that  the  track-up  map  display  actually  supported  better  global  situation 
awareness  than  the  north-up  map.  This  was  seen  by  the  increased  speed  at  which  pilots  were 
able  to  identify  a  target's  location  in  the  FFOV  when  using  the  track-up  map,  an  advantage  which 
they  attributed  to  the  consistent  ERF  shared  between  the  FFOV  and  the  rotating  map  This 
ability  to  ascertain  a  target's  location  is  essential  to  maintaining  maximum  hazard  awareness. 


Also,  this  study  found  no  evidence  supporting  previous  research  findings  that  the  fixed  map 
supported  better  long-term  retention  of  terrain  features. 
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Egocentric  versus  exocentric  viewpoint 

As  discussed  previously,  the  location  of  the  viewpoint  determines  whether  an  electronic  map 
display  presents  an  immersed  FOV  (panel  A  and  B,  Figure  1)  or  an  exocentric  FOV  (panel  B, 
Figure  1).  Many  researchers  believe  that  these  two  different  3D  viewpoints  are  as  different  from 
each  other  as  both  are  from  a  2D  display  (Olmos  et  al.,  1998).  Several  studies  have  compared 
these  two  viewpoints,  sometimes  along  with  a  comparison  of  a  2D  coplanar  map. 

Barfield,  Rosenberg,  &  Furness  (1995)  compared  an  egocentric  and  an  exocentric  electronic 
map  display  in  the  context  of  a  simulated  F-16  scenario.  The  authors  measured  the  subjects 
deviations  from  the  flight  path  and  found  that  the  immersed  display  resulted  in  superior  local 
guidance  performance,  which  they  attributed  to  the  compatibility  of  motion  between  the 
simulated  out-the-window  view  and  the  immersed  display.  Global  situation  awareness  was  also 
measured  by  using  a  posttest  spatial  reconstruction  task  that  required  the  subjects  to  locate  the 
position  of  each  target  on  a  computer  generated  top-down  map  of  the  flight  scene.  In  contrast  to 
local  guidance,  the  results  for  this  task  revealed  far  better  performance  by  the  exocentric  display. 
The  authors  concluded  that  this  advantage  resulted  from  the  static  compatibility  that  occurred 
between  the  exocentric  display  and  the  target  positioning  task  planview  map. 

Another  study  by  Wickens  and  Prevett  (1995)  compared  five  different  map  display 
viewpoints,  consisting  of  an  immersed  display  with  an  expanded  geometric  field  of  view 
(GFOV),  three  different  exocentric  displays  (varied  by  distance  vector  lengths),  and  a  2D 
coplanar  display.  Examination  of  flightpath  deviations  revealed  that  the  immersed  display 
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resulted  in  superior  horizontal  and  vertical  flight  performance.  The  researchers  also  measured 
global  situation  awareness  by  incorporating  a  freeze  into  the  scenario  to  query  the  subjects  as  to 
their  awareness  of  the  situation  at  that  moment,  and  by  administering  a  posttest  map 
reconstruction  task.  Results  for  the  freeze  task  revealed  that  performance  for  the  egocentric 
display  was  substantially  worse  than  the  other  map  display  types,  while  the  map  reconstruction 
task  did  not  reveal  any  significant  differences  between  the  display  types.  The  results  for  the 
freeze/query  measure  were  attributed,  in  part,  to  the  distortion  of  the  displayed  world  required  to 
bring  all  objects  in  front  of  the  aircraft  into  the  immersive  view  (i.e.,  distortion  caused  by  the 
expanded  GFOV),  while  the  map  reconstruction  task  results  were  connected  to  the  possibility 
that  terrain  information  was  only  needed  to  sustain  ongoing  situation  awareness,  and  therefore  it 
was  not  encoded  into  long-term  memory. 

Olmos  et  al.  (1998)  compared  immersed,  exocentric,  and  2D  coplanar  map  displays  in  an 
experiment  that  required  subjects  to  navigate  through  a  simulated  airspace  populated  by 
waypoints,  hazards,  and  traffic.  As  with  Barfield  et  al.  (1995)  and  Wickens  and  Prevett  (1995), 
the  researchers  found  that  the  immersed  display  proved  to  be  superior  for  local  guidance.  Olmos 
et  al.  (1998)  also  incorporated  a  number  of  global  situation  awareness  measures,  to  include  actual 
contact  with  surrounding  terrain  or  hazards,  verbal  report  of  external  threats,  and  an  implicit 
measure  resulting  from  pop-up  conflicts.  Results  from  these  measures  were  somewhat 
ambiguous,  revealing  that  performance  with  the  immersed  display  (which  had  also  incorporated 
a  small  exocentric  hazard  display  for  global  situation  awareness)  suffered  from  visual  attention 
allocation  problems.  Those  problems  were  attributed  to  the  more  "compelling"  and  information 
rich  immersed  view  drawing  attention  away  from  the  hazard  display.  Also,  both  of  the  3D 
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displays  suffered  in  regard  to  hazard  awareness,  because  of  problems  associated  with  estimating 
the  height  of  approaching  hazards. 

Thus,  there  appears  to  be  a  trade-off  with  respects  to  the  benefits  of  the  egocentric  viewpoint. 
While  immersed  displays  provide  superior  performance  for  local  guidance  and  navigation 
requirements,  they  suffer  in  their  ability  to  provide  sufficient  global  situation  awareness. 

Perspective  versus  planar  viewpoint 

The  attitude,  or  elevation  angle,  of  the  viewpoint  determines  whether  the  map  display  will 
have  a  perspective  view  or  a  planar  view.  As  discussed  before,  the  elevation  angle  may  be 
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presented  from  various  vertical  perspectives  between  the  range  of  0  and  -90  ,  with  the  planar 

o 

view  (panel  D,  Figure  1)  occurring  when  the  viewpoint  elevation  angle  is  at  -90  .  Because  this 
view  results  in  the  total  loss  of  vertical  resolution,  this  2D  "God's  eye"  view  is  often  coupled  with 

a  0°  vertical  display,  resulting  in  a  co-planar  view  display  (Wickens,  1998).  There  have  been 
several  studies  that  have  evaluated  how  these  two  types  of  views  compare. 

Merwin,  O'Brien  and  Wickens  (1997)  compared  exocentric  3D  and  2D  coplanar  versions  of  a 
cockpit  display  of  traffic  information  (CDTI),  that  involved  pilots  flying  a  series  of  conflict 
avoidance  maneuvers.  The  researchers  measured  the  pilot's  ability  to  quickly  fly  to  a  distant 
waypoint  while  avoiding  actual  or  predicted  traffic  conflicts  (analogous  to  hazard  awareness). 
The  results  indicated  a  consistent  advantage  for  the  2D  coplanar  display  over  the  exocentric  3D 
format  in  supporting  traffic  avoidance  maneuvers.  These  results  were  attributable  to  the 
ambiguity  with  which  the  exocentric  display  depicts  position  and  separation  along  the  line  of 
sight  (LOS). 
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In  the  study  by  Rate  and  Wickens  (1993)  requiring  pilots  to  fly  a  curved  approach  to  landing 
in  a  terminal  area,  information  was  rendered  on  either  an  integrated,  exocentric  3D  display  or  a 
2D  co-planar  display.  The  results  of  this  study  revealed  that  the  2D  display  was  superior  to  the 
3D  view  in  terms  of  both  lateral  and  vertical  tracking.  This  result  was  attributed  in  part  to  the 
ambiguity  of  depicting  spatial  information  on  a  3D  display.  Measurement  of  global  situation 
awareness  was  assessed  in  terms  of  error  and  response  latency  to  probed  questions  regarding 
terrain  features.  This  measure  of  hazard  (terrain)  awareness  provided  ambiguous  results  as  to 
which  display  viewpoint  provided  superior  GSA. 

Wickens  et  al.  (1996)  carried  out  an  experiment  using  a  similar  paradigm  to  that  of  Rate  and 
Wickens  (1993),  however,  they  improved  some  of  the  deficiencies  in  the  presentation  of  altitude 
on  the  3D  display.  Results  from  this  study  showed  lateral  tracking  to  be  equivalent  between  the 
two  views,  and  only  a  small  3D  cost  in  terms  of  vertical  tracking,  which  showed  a  trend  toward 
substantial  improvement  with  practice.  The  authors  used  a  position  report  task,  a  frozen-screen 
test,  and  a  map  reconstruction  task  to  assess  the  level  of  global  situation  awareness.  Again,  the 
global  awareness  measures  revealed  ambiguous  results,  with  the  3D  display  allowing  for  faster 
responses  to  the  global  awareness  questions,  but  the  2D  display  providing  more  accurate 
responses.  Results  from  the  map  reconstruction  task  revealed  a  marginally  significant  difference 
between  drawings  based  upon  the  two  maps,  with  maps  drawn  on  the  basis  of  2D  map  exposure 
containing  slightly  more  accurate  placement  of  terrain  features,  as  might  be  anticipated  given  the 
LOS  ambiguity  of  the  3D  display. 

The  Wickens  and  Prevett  (1995)  study  discussed  above  also  compared  3D  exocentric  displays 
with  a  2D  coplanar  display.  In  this  study,  the  three  exocentric  displays  varied  with  regards  to  the 
length  of  the  viewing  distance  vector,  with  the  three  different  lengths  being  3,000  m,  7,500  m. 
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and  21,000  m.  Tracking  performance  results  revealed  that  horizontal  tracking  suffered  with  the 
far  exocentric  display  (i.e.,  21,000  m  viewing  distance  vector),  while  vertical  tracking  suffered 
with  use  of  the  close  exocentric  display  (i.e.,  3,000  m).  Hence  the  middle  distance  (7,500  m) 
tended  to  be  optimal.  The  researchers  also  measured  global  situation  awareness  by  incorporating 
a  freeze  into  the  scenario  to  query  the  subjects  as  to  their  awareness  of  the  situation  at  that 
moment  (broken  down  into  world-referenced  questions  and  ego-referenced  questions),  and  by 
administering  a  posttest  map  reconstruction  task.  Results  for  the  map  reconstruction  measure  did 
not  reveal  any  significant  differences  between  the  different  types  of  map  displays,  while  the 
query  questions  provided  ambiguous  results.  Performance  with  the  coplanar  display  was  slower 
and  less  accurate  for  the  world-referenced  questions,  but  more  accurate  for  the  ego-referenced 
questions. 

Another  study  that  compared  an  exocentric  3D  display  with  a  2D  coplanar  display  was  the 
Olmos  et  al.  (1998)  study  discussed  above,  where  subjects  flew  through  a  simulated  airspace 
populated  by  waypoints,  hazards,  and  traffic.  Results  revealed  that  both  the  exocentric  and  the 
2D  displays  suffered  in  terms  of  navigation  support,  however,  each  for  a  different  reason.  The 
exocentric  display  suffered  ambiguity  in  estimating  ownship  position  and  heading,  while  the  2D 
display  suffered  from  the  costs  of  lateral  and  vertical  cross  panel  integration.  In  depth  analyses 
revealed  differential  costs  for  the  coplanar  and  exocentric  displays  on  vertical  versus  lateral 
flight  legs,  with  the  coplanar  display  suffering  the  greatest  costs  on  legs  involving  only  vertical 
maneuvers  and  the  exocentric  display  suffering  the  greatest  costs  on  lateral  maneuvering  legs.  A 
number  of  global  situation  awareness  measures  were  also  incorporated  into  this  study,  to  include 
verbal  report  of  external  threats,  actual  contact  with  surrounding  terrain  or  hazards,  and  an 
implicit  measure  resulting  from  pop-up  conflicts.  Comparison  of  these  GSA  measures  between 
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the  exocentric  and  2D  displays  revealed  ambiguous  results  similar  to  those  found  by  Wickens  et 
al.  (1996),  with  the  exocentric  display  providing  quicker  response  times  for  the  verbal  report  of 
external  threats  and  reduced  time  spent  in  contact  with  hazard  volumes,  while  the  2D  coplanar 
display  provided  more  accurate  judgement  of  the  altitude  of  pop-up  conflicts. 

Olmos  et  al.  (1997)  reexamined  the  variables  manipulated  by  Wickens  et  al.  (1996),  however 
they  incorporated  a  FFOV  to  support  the  pilot's  navigational  task,  which  simulated  VMC.  In 
contrast  to  the  results  of  previous  studies  (Rate  &  Wickens,  1993;  Wickens  et  al.  1996),  this 
study  did  not  observe  a  cost  for  vertical  tracking  with  the  3D  display.  In  addition,  the  authors 
found  a  slight  trend  toward  better  lateral  tracking  performance  with  the  3D  display  as  compared 
to  the  coplanar  display.  They  attributed  these  results  to  a  reduced  scanning  cost  for  the  3D 
display  when  transitioning  between  the  FFOV  and  the  electronic  map,  to  an  adherence  to 
Roscoe's  (1968)  principle  of  pictorial  realism,  and  to  further  augmentations  to  improve  the 
altitude  coding  on  the  3D  display.  However,  the  3D  advantage  in  terms  of  flight  control  was  not 
replicated  for  their  global  situation  awareness  measures  of  a  position  report  task,  a  FFOV-map 
comparison  task,  a  direction-indicating  task,  and  map  reconstruction  task.  The  FFOV-map 
comparison  task  and  altitude  probes  within  the  position  report  task  revealed  significant 
differences  between  the  two  views,  and  both  found  the  2D  format  to  be  superior  in  terms  of 
global  situational  awareness. 

One  of  the  reasons  why  there  may  be  ambiguity  concerning  which  of  these  two  displays 
provides  the  best  local  guidance  and  global  situation  awareness  support  may  be  because  of  the 
different  costs  associated  with  each  type  of  display.  Exocentric  displays  are  beneficial  in  that 
they  reduce  the  need  for  visual  scanning  between  panels,  they  reduce  the  need  for  mental 
integration,  and  they  present  a  3D  "picture"  that  looks  like  the  real  visual  scene  viewed  by  the 
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pilot  (Wickens  et  al,  1996).  However,  against  these  potential  advantages  are  some  related  costs. 
Exocentric  displays  typically  create  some  ambiguity  regarding  the  precise  position  of  an  object 
along  the  line  of  sight,  they  typically  lose  resolution  in  depicting  motion  or  position  along  the 
viewing  axis,  and  imprecise  matching  between  the  viewing  distance  between  the  display  surface 
and  the  pilot's  eyes  may  result  in  distortion  in  perceived  location  of  objects  (Merwin  et  al.,  1997; 
McGreevy  &  Ellis,  1986;  Olmos  et  al.,  1998;  Wickens  et  al.,  1996).  Therefore,  the  relative  value 
of  either  type  of  display  may  be  determined  by  the  need  to  perform  certain  tasks  that  may  or  may 
not  impose  the  associated  costs  of  each. 

Besides  the  type  of  tasks  that  pilots  are  required  to  perform,  the  altitude  of  flight  may  also 
determine  which  display  is  more  effective.  This  dependency  was  revealed  by  a  recent  study  by 
Hickox  and  Wickens  (1997),  that  evaluated  the  effect  of  disparity  between  a  range  of  FFOV 
images  from  different  elevation  angles  and  an  exocentric  map  display  also  presented  from 
different  elevation  angles.  The  map  display  and  the  FFOV  were  simultaneously  presented  to  the 
subjects,  and  a  same/different  comparison  task  was  used  to  determine  the  effect  of  varying 
degrees  of  elevation  angle  disparity  between  the  FFOV  and  the  exocentric  map  display. 

Response  times  for  the  comparison  task  revealed  that  small  disparities  between  the  FFOV  and 
map  display,  across  most  map  display  elevation  angles,  resulted  in  negligible  costs,  however, 
larger  disparities  resulted  in  the  cost  of  a  particular  disparity  being  amplified.  These  results 
suggest  that  the  advantage  for  a  tethered  exocentric  display,  where  the  camera  elevation  angle  is 
slaved  to  the  aircraft's  pitch  or  attitude  relative  to  a  2D  display,  may  increase  at  lower  flight 
levels  when  the  pilot's  view  of  the  terrain  is  more  forward  looking. 

An  experiment  similar  to  Hickox  and  Wickens  (1997)  was  conducted  by  Schreiber,  Wickens, 
Renner,  Alton,  and  Hickox  (1998),  which  also  utilized  static  3D  images  for  both  the  map  and  the 
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FFOV,  with  several  different  elevation  angles  used  for  both  map  and  FFOV  images.  However, 
instead  of  presenting  the  map  display  and  FFOV  simultaneously,  the  subjects  were  presented 
with  the  3D-map  image  for  5  seconds  after  which  the  computer  would  switch  to  the  FFOV 
picture.  As  soon  as  the  FFOV  image  was  displayed,  the  subjects  were  asked  to  judge  whether 
the  map  and  the  FFOV  matched  or  mismatched,  with  both  response  time  and  accuracy  recorded. 
The  researchers  found  that  response  time  increased  and  accuracy  of  performance  decreased  as 
the  foreshortening  disparity  increased,  where  foreshortening  disparity  captures  the  effect  of 
different  amounts  of  elevation  angular  disparity  at  different  map  angles.  The  researchers 
concluded  that  at  low  approach  angles  (i.e.,  low  altitude  flight),  small  to  medium  levels  of 
elevation  angular  disparity  result  in  large  foreshortening  disparities,  which  correspond  to 
decreased  performance. 

This  section  has  shown  that  there  are  many  different  factors  that  may  influence  the 
effectiveness  of  displays  using  either  exocentric  viewpoints  or  2D  viewpoints,  with  the  type  of 
task  and  altitude  of  flight  particularly  affecting  their  usefulness.  It  is  also  worth  noting  the 
continued  ambiguous  findings  regarding  which  of  the  two  display  viewpoints  best  facilitates 
situation  awareness. 

Present  research 

In  summary,  the  literature  has  shown  that  there  are  several  different  parameters  whose 
influences  must  be  evaluated  when  determining  which  frame  of  reference  an  electronic  map 
display  should  adopt.  These  parameters  determine  how  much  ERF  and  WRF  information  a 
particular  map  display  presents  to  the  pilot.  The  level  of  ERF  and  WRF  knowledge  that  the  map 
display  facilitates  has  a  direct  influence  on  a  pilot's  ability  to  perform  local  guidance  tasks  and 
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achieve  and  maintain  global  situation  awareness.  We  will  now  summarize  the  previous  research 
findings  and  highlight  particular  concerns  for  the  low  altitude  rotorcraft  environment. 

Previous  research  has  shown  the  utility  of  a  track-up  map  format.  Several  studies  have 
shown  that  rotating  maps  produce  greater  flight  path  tracking  accuracy  and  navigational 
efficiency  (Aretz,  1991;  Andre,  Wickens,  Moorman,  &  Boschelli,  1991;  Olmos  et  al.,  1997;  Rate 
&  Wickens,  1993;  Wickens  et  al.,  1996).  This  characteristic  makes  track-up  maps  preferable 
when  flight  profiles  require  rapid  decisions  regarding  lateral  turns  (Wickens  et  al.,  1996). 
Referring  back  to  Flach  and  Warren's  (1995)  precision  and  responsiveness  requirements  for  low 
altitude  flight,  this  research  shows  that  a  track-up  map  format  seems  appropriate  for  use  in  the 
rotorcraft  context.  Also,  recent  work  by  Olmos  et  al.  (1997)  has  shown  that  a  track-up  map 
display  may  actually  improve  hazard  awareness  and  global  situation  awareness  when  used  during 
VMC.  Since  over  93%  of  rotorcraft  missions  occur  during  VMC,  a  rotating  map  would  seem  to 
be  desirable  for  rotorcraft  operations.  Therefore,  considering  the  effectiveness  of  the  track-up 
format  to  support  both  local  guidance  and  global  situation  awareness,  we  chose  to  employ  a 
rotating  map  display  for  this  study. 

The  literature  does  not  show  the  same  degree  of  agreement  concerning  the  use  of  immersed, 
exocentric,  or  2D  coplanar  map  viewpoints  and  their  effect  on  local  guidance  and  GSA.  There 
seems  to  be  a  trade-off  between  the  different  types  of  displays  and  the  level  of  local  guidance 
and  GSA  each  offers.  The  immersed  display  has  been  shown  to  offer  superior  local  guidance 
performance  (Barfield  et  al.,  1995;  Olmos  et  al.,  1998;  Wickens  &  Prevett,  1995),  however  it 
also  suffers  from  a  "keyhole"  view  of  the  world,  which  makes  obtaining  sufficient  GSA  difficult 
(Wickens,  1998;  Woods,  1984).  Efforts  to  reduce  this  "keyhole"  cost  by  expanding  the  GFOV 
results  in  a  distorted  view  of  the  displayed  world  which  in  itself  makes  obtaining  GSA  difficult 
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(Wickens  &  Prevett,  1995).  The  differences  between  the  exocentric  and  2D  coplanar  formats  are 
more  subtle  and  depend  on  a  tradeoff  between  the  costs  imposed  by  each.  The  coplanar  format 
imposes  visual  scanning  requirements  and  mental  transformations  to  cognitively  integrate 
information  across  the  lateral  and  vertical  displays  (Wickens  et  ah,  1996),  while  the  exocentric 
format  suffers  from  a  loss  of  resolution  and  ambiguity  and  imprecision  in  judgements  along  the 
line  of  sight  (Wickens  et  ah,  1996).  The  relative  importance  of  these  costs,  and  thus  the  value  of 
either  type  of  display  format,  may  be  determined  by  the  cost  of  visual  scanning  requirements 
versus  the  need  for  dimensional  integration  within  the  task  and  the  need  for  precise  judgements 
along  axes  that  are  parallel  to  the  viewing  axis  within  the  3D  display  (Wickens,  1998). 

Therefore,  it  is  somewhat  unclear  as  to  which  of  these  three  displays  most  effectively  supports 
both  local  guidance  and  global  situation  awareness  (Wickens,  1998). 

This  uncertainty  concerning  which  display  is  most  effective  for  providing  both  local  guidance 
and  global  situation  awareness  is  an  important  consideration  if  electronic  map  displays  are  to  be 
utilized.  However,  there  are  many  questions  that  the  previous  research  does  not  address  in 
regards  to  the  use  of  electronic  map  displays  in  a  rotorcraft  cockpit.  Most  of  the  research  on 
electronic  map  displays  has  been  conducted  in  fixed  wing  aircraft  scenarios  where  the  pilots  are 
flying  high  over  the  terrain  and  do  not  have  to  worry  about  terrain  or  obstacle  strikes  (Barfield  et 
al.,  1995;  Olmos  et  al.,  1997;  Merwin  et  al.,  1997;  Rate  &  Wickens,  1993;  Wickens  &  Prevett, 
1995;  Wickens  et  al.,  1996).  As  mentioned  in  the  introduction,  the  versatility  of  the  helicopter 
often  results  in  it  performing  its  missions  at  very  low  altitudes.  Therefore,  research  needs  to  be 
conducted  in  a  low  altitude  rotorcraft  specific  context  to  see  if  the  conclusions  drawn  from  these 
previous  studies  generalize  to  very  low  altitude  flight. 
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Another  shortcoming  of  the  previous  research  is  that,  with  the  exception  of  Olmos  et  al. 
(1997),  the  majority  of  the  studies  investigating  FOR  have  been  done  in  instrument  flight  rules 
scenarios  in  which  no  FFOV  was  presented.  Also,  even  though  Olmos  et  al.  (1997)  incorporated 
a  FFOV  into  their  study,  they  only  evaluated  2D  coplanar  and  3D  exocentric  displays.  Recent 
research  by  Hickox  and  Wickens  (1997)  suggests  that  a  3D  immersed  display  may  be 
particularly  advantageous  for  very  low  altitude  flights.  Therefore,  since  many  rotorcraft 
missions  occur  at  very  low  altitudes  during  VMC,  it  seems  appropriate,  and  necessary,  to 
evaluate  the  effects  that  immersed,  exocentric,  and  2D  displays  have  on  local  guidance  and  GSA 
during  low  altitude  flights  when  a  FFOV  is  present. 

Furthermore,  although  many  studies  have  shown  that  an  immersed  display  suffers  from  a 
"keyhole"  view  of  the  world,  which  makes  obtaining  sufficient  GSA  difficult  (Wickens  & 
Prevett,  1995;  Wickens,  1998),  the  tradeoff  between  exocentric  and  2D  displays  in  terms  of  GSA 
has  remained  ambiguous  (Rate  &  Wickens,  1993;  Olmos  et  al.,  1998;  Wickens  et  al.,  1996). 

This  ambiguity  suggests  that  more  research  needs  to  be  conducted  in  this  area  to  try  and  resolve 
the  uncertainty  that  exists  as  to  which  of  the  two  displays  provides  the  best  GSA.  Finally,  even 
though  many  of  the  cited  studies  in  the  FOR  section  used  SAGAT-like  measures  of  GSA,  none 
of  them  utilized  the  SART  (Taylor,  1990)  or  implicit  performance  measures.  These  two  GSA 
measures  need  to  be  evaluated  in  regards  to  how  effectively  they  measure  the  level  of  GSA 
facilitated  by  the  different  types  of  electronic  map  displays  when  used  in  a  low  altitude  scenario. 

Based  on  these  research  questions,  the  present  study  will  evaluate  pilot  performance  using  an 
immersed,  exocentric,  or  2D  display  at  two  different  altitudes  while  flying  with  a  simulated 
FFOV  through  multiple  rotorcraft-specific  scenarios  that  incorporate  numerous  man-made  and 
natural  hazards.  Pilot  performance  will  be  assessed  by  measures  of  local  guidance  ability  and 
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global  situation  awareness.  The  simulation  computers  will  be  able  to  measure  local  guidance 
ability  as  the  deviations  from  the  defined  flight  path  for  each  mission.  Also,  we  will  utilize  three 
measures  of  GSA,  since  several  different  measures  need  to  be  utilized  to  evaluate  pilot  GSA,  and 
then  evaluated  as  to  the  degree  to  which  they  tend  to  agree  or  disagree  in  regards  to  a  specific 
display's  ability  to  promote  situation  awareness  (Endsley,  1996).  Based  on  Endsley's  (1996) 
review  of  possible  measurement  techniques,  we  chose  to  incorporate  three  different  measures  of 
GSA,  to  include:  1)  an  implicit  performance  measure,  achieved  by  programming  situations  into 
the  scenarios  (i.e.,  crosswinds  during  hover)  that  allowed  us  to  infer  the  subjects  level  of  GSA,  2) 
a  subjective  measure,  in  this  case  the  SART  (Taylor,  1990),  and  3)  a  freezing  technique,  similar 
to  SAGAT  (Endsley,  1987),  which  evaluated  the  pilots  knowledge  level  of  the  3D  airspace. 

Considering  the  independent  variables  being  manipulated,  our  dependent  measures,  and  the 
previous  research,  we  developed  three  hypotheses  for  this  study.  The  first  hypothesis  is  that  the 
egocentric  display  will  provide  the  best  local  guidance  performance,  especially  during  the  low 
altitude  scenarios  as  the  display  becomes  more  congruent  with  the  FFOV  (Barfield  et  al.,  1995; 
Olmos  et  al.,  1998;  Wickens  &  Prevett,  1995).  However,  we  feel  the  exocentric  display  will 
provide  equivalent  performance  during  the  low  altitude  flight,  because  it  will  also  provide  a 
viewing  perspective  that  is  fairly  congruent  with  the  pilot's  direct  view  of  the  forward  terrain 
(Hickox  &  Wickens,  1997).  The  second  hypothesis  is  that  the  exocentric  display  will  provide 
the  best  performance  on  the  global  situation  awareness  measures.  We  believe  this  advantage  will 
result  because  the  egocentric  display  suffers  from  a  "keyhole"  view  of  the  world,  which  is  poor 
for  obtaining  GSA  (Wickens,  1998,  Woods,  1984).  Also,  because  of  the  need  to  cross  check 
between  the  FFOV  and  the  map  display,  we  feel  the  fairly  congruent  view  of  the  FFOV  provided 
by  the  exocentric  display  will  outweigh  its  associated  costs,  which  will  result  in  better 
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performance  than  the  2D  display.  Finally,  our  third  hypothesis  is  that  the  SART  (Taylor,  1990) 
will  be  a  poor  measure  of  the  actual  level  of  GSA  achieved  by  the  pilots.  We  feel  this  problem 
will  result  because  the  pilots  may  have  a  hard  time  determining  their  own  inaccuracies  or  judging 
information  that  they  are  not  aware  of,  and  therefore  they  will  have  a  limited  basis  for  making  a 
judgement  of  GSA  with  this  measure  (Endsley,  1996;  Endsley  et  al.,  1998).  As  to  the  GSA 
measures,  there  is  considerable  variability  among  researchers  as  to  their  recommendations 
regarding  which  measures  should  be  used,  therefore,  a  further  goal  of  the  present  study  will  be  to 
evaluate  the  degree  to  which  the  different  measures  agree  or  disagree  with  regards  to  the  ability 
of  the  different  displays  to  facilitate  global  situation  awareness. 
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METHODS 

Subjects 

Eighteen  students  enrolled  in  or  teaching  flight  classes  for  the  University  of  Illinois  Institute 
of  Aviation  participated  in  the  experiment,  with  the  subjects  ranging  from  20  to  35  years  of  age 
(mean=22.5).  All  subjects  were  pilots  with  at  least  a  private  license  and  some  instrument  time, 
with  a  range  of  flight  time  from  145  to  3250  hours  (mean=433).  Excluding  the  one  subject  with 
3250  flight  hours  reduced  the  range  of  flight  time  from  145  to  600  hours  (mean=267).  All 
subjects  received  the  same  instructions  (see  Appendix  A)  and  were  paid  $6.00  per  hour  for  their 
participation.  An  additional  $20.00  bonus  was  awarded  to  the  top  performer  and  two  $1 0.00 
bonuses  were  awarded  to  the  two  next  best  performers.  The  winners  of  the  bonus  money  were 
determined  post-experiment  by  rank-ordering  each  subject's  performance  on  several  of  the 
performance  measures  and  then  averaging  across  the  ranks  for  an  overall  score  for  each  subject. 

Apparatus 

The  study  was  conducted  on  an  Evans  and  Sutherland  (EandS)  SPX  500T  image  generator 
driving  two  Electrohome  ECP  3000  color  projectors,  and  a  Silicon  Graphics  IRIS  4D-70GT 
workstation  driving  a  44-cm  diagonal  color  monitor.  The  EandS  was  used  to  display  the  forward 
field  of  view  to  the  subjects  to  simulate  visual  navigation.  Two  projection  screens,  each 
measuring  228.6  cm  vertically  and  304.8  cm  horizontally,  were  set  at  an  angle  of  1 15°  to  each 
other.  One  screen  was  located  300  cm  directly  in  front  of  the  participant  and  the  other  was  offset 
to  the  left  side,  providing  a  continuous  38  0  vertical  by  1 12  °  horizontal  field  of  view.  The  scene 
detail  of  the  EandS  was  a  relatively  realistic  depiction  of  mountainous  terrain,  viewed  on  a  hazy 


day  and  from  an  elevation  and  azimuth  angle  dependent  on  the  pilot's  momentary  position  and 
orientation  with  the  world.  Update  rate  for  the  EandS  system  was  50  Hz. 
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The  IRIS  workstation  was  used  to  display  the  electronic  maps  during  the  scenarios,  and  it  was 
located  directly  in  front  of  the  participants  such  that  it  did  not  interfere  with  the  participants' 
view  of  the  EandS  projector  screen.  The  workstation  depicted  an  attitude-direction  indicator 
(ADI),  a  compass  during  defined  periods  of  the  scenario,  speed,  on-screen  heading  and  an 
altitude  display,  which  showed  altitude  above  sea  level  (see  Figure  x).  The  view  on  the  IRIS 
map  was  schematic  in  appearance  and  was  depicted  from  a  fixed  elevation  angle,  which  varied 
with  experimental  condition,  and  azimuth  angle  as  described  in  more  detail  later.  The  screen 
update  rate  for  the  IRIS  display  was  approximately  5  Hz. 

A  Microsoft  Sidewinder  3D  Pro  joystick  was  attached  to  the  right  arm  of  the  participant's 
chair,  which  allowed  for  different  control  inputs  and  dynamics  according  to  whether  the 
participant  was  in  forward  flight  mode  or  hover  mode.  The  paradigm  for  this  study  was  that  the 
aircraft  was  considered  in  hover  mode  only  when  the  airspeed  equaled  zero.  The  "china  hat"  on 
the  joystick  controlled  airspeed,  and  therefore,  it  also  controlled  the  transition  into  and  out  of 
hover  mode.  During  forward  flight,  the  joystick  controlled  the  lateral  and  vertical  axes  of  the 
aircraft,  with  standard  aviation  dynamics.  Pushing  forward/back  on  the  stick  caused  a 
decrease/increase  in  pitch,  with  the  rate  of  change  of  altitude  directly  controlled  by  the  amount  of 
pitch.  Moving  the  joystick  to  the  right  or  left  made  the  helicopter  roll  right  or  left,  respectively, 
with  the  roll  angle  directly  proportional  to  the  rate  of  change  of  heading.  During  hover  mode, 
pushing  the  joystick  in  any  direction  caused  the  helicopter  to  slew,  or  translate,  in  that  direction, 
while  maintaining  the  current  aircraft  heading.  For  example,  pushing  the  joystick  forward 
caused  the  helicopter  to  translate  (move)  forward,  but  did  not  result  in  a  decrease  in  pitch. 
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Likewise,  moving  the  joystick  to  the  right  caused  the  helicopter  to  translate  to  the  right,  but  did 
not  result  in  a  rightward  roll  or  a  change  in  heading.  The  "Twist  grip"  on  the  joystick  was  used 
during  hover  mode  to  change  the  heading  (yaw),  with  a  twist  to  the  left  causing  the  aircraft  to 
turn  to  the  left,  or  decrease  heading,  and  a  twist  to  the  right  causing  the  aircraft  to  turn  to  the 
right,  or  increase  heading.  Also,  the  top  side  and  bottom  side  buttons  allowed  the  helicopter  to 
ascend  or  descend  during  hover  mode. 

Simulation  database 

The  flight  area  used  for  this  study,  the  Freemont  region,  was  specifically  developed  for 
studies  at  the  University  of  Illinois,  and  the  database  was  not  modeled  after  any  actual 
geographic  region.  The  Freemont  region  was  rectangular  in  shape  and  measured  approximately 
37.5  nautical  miles  by  29.5  nautical  miles.  Also,  the  region  contained  a  diverse  range  of  terrain 
types. 

Electronic  map  displays 

There  were  three  different  display  conditions  used  in  the  study.  In  each  of  the  displays,  the 
world  remained  the  same,  while  the  variation  affected  only  the  view  of  the  world  presented  to  the 
subjects.  Since  a  rotating  map  was  desired  for  the  study,  the  viewpoint  rotation  was  slaved  to  the 
aircraft  for  all  of  the  display  conditions.  The  following  descriptions  define  the  different 
conditions  in  greater  detail. 

The  egocentric  display,  as  seen  in  Figure  2.1a  on  page  44,  was  presented  from  the  perspective 
of  the  subject  sitting  in  the  pilot's  seat.  The  field  of  view  (FOV)  was  set  at  45  in  the  horizontal 
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and  36°  in  the  vertical.  Also,  since  the  viewpoint  location  was  positioned  within  the  aircraft, 
azimuth  angle  and  elevation  angle  were  zero. 

The  exocentric  display  (see  Figure  2.1b,  page  44)  was  presented  from  the  perspective  of  the 
subject  viewing  the  aircraft  from  above  and  behind.  The  azimuth  angle  for  this  display  was  set  at 

0°,  or  directly  behind  the  aircraft,  and  the  elevation  angle  was  set  at  45  degrees.  The  distance 
from  the  viewpoint  to  the  aircraft  was  3 1 ,000  feet,  and  the  FOV  for  this  display  was  45  in  the 
horizontal  and  36°  in  the  vertical. 

The  planar  display  (see  Figure  2.1c,  page  45)  was  presented  from  the  perspective  of  the 
subject  looking  directly  down  on  the  aircraft.  The  elevation  angle  was  set  at  -90  ,  which  caused 

O 

complete  compression  along  the  vertical  axis,  and  the  azimuth  angle  was  set  at  0  . 

Experimental  design 

This  experiment  manipulated  two  variables  and  was  carried  out  using  a  3  x  2  block,  within 
subjects  design.  The  variables  are: 

1 .  3D  egocentric  versus  3D  exocentric  versus  2D  planar,  and 

2.  Low  altitude  flight  (200  ft)  versus  high  altitude  flight  ( 1 000  ft). 

All  participants  flew  both  a  low  altitude  and  a  high  altitude  scenario  with  each  of  the  three 
display  types  (egocentric,  exocentric,  2D),  for  a  total  of  six  missions.  The  display  ordering  was 
blocked  and  allowed  for  both  a  low  altitude  and  a  high  altitude  scenario  to  be  flown  within  a 
given  display  type.  Subjects  were  assigned  to  one  of  six  possible  groups,  shown  in  Table  2.1, 
which  were  defined  by  the  ordering  of  display  type  presentation  blocks  and  low  altitude  missions 
(paths  1,2,3)  and  high  altitude  missions  (paths  4,5,6)  presentation.  The  ordering  of  the  display 
type  blocks  and  low  and  high  altitude  scenarios  was  counterbalanced  across  groups. 
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Practice 

Scored  Scenarios 

1  2  3 

1  2  3  4  5  6 

Group 

Full 

2mln 

2min 

Map/Path 

Map/Path 

Map/Path 

Map/Path 

Map/Path 

Map/Path 

1 

2D 

Exocentric 

Egocentric 

Egocentric/4 

Egocentric/1 

Exocentric/2 

Exocentric/5 

2D/6 

2D/3 

2 

Exocentric 

Egocentric 

2D 

2D/2 

2D/5 

Egocentric/6 

Egocentric/3 

Exocentric/1 

Exocentric/4 

3 

Egocentric 

2D 

Exocentric 

Exocentric/6 

Exocentric/3 

2D/1 

2D/4 

Egocentric/5 

Egocentric/2 

4 

Egocentric 

Exocentric 

2D 

2D/1 

2D/6 

Exocentric/5 

Exocentric/2 

Egocentric/3 

Egocentric/4 

5 

Exocentric 

2D 

Egocentric 

Egocentric/5 

Egocentric/2 

2D/3 

2D/4 

Exocentric/6 

Exocentric/1 

6 

2D 

Egocentric 

Exocentric 

Exocentric/3 

Exocentric/4 

Egocentric/6 

Egocentric/1 

2D/2 

2D/5 

Table  2.1:  Experimental  design  used  in  this  study. 


Procedure  and  task 

The  trials  were  broken  down  into  two  one-hour  sessions  during  the  same  day.  At  the 
beginning  of  the  first  session,  the  pilots  were  given  written  instructions  that  described  the  tasks 
and  goals  of  the  experiment.  The  pilots  were  then  allowed  a  fifteen-minute  practice  session, 
during  which  they  were  introduced  to  the  handling  and  control  characteristics  of  the  simulation. 
This  practice  session  utilized  a  short  path  that  included  all  of  the  unique  situations  and  hazards 
that  the  pilot  would  experience  during  each  trial.  The  subjects  flew  the  practice  path  using  one 
type  of  map  display,  taking  approximately  ten-minutes,  then  had  two  two-minute  segments  that 
exposed  the  subjects  to  the  other  two  map  display  types.  Which  map  display  was  used  for  the 
full  practice  mission  depended  on  the  group  to  which  the  subject  was  assigned  (refer  to 
Table  2.1).  Subjects  were  instructed  that  during  the  self  paced  practice  session,  minimizing  the 
deviations  from  the  flightpath  was  not  the  most  important  issue,  but  rather  the  main  purpose  was 
familiarization  with  the  unique  control  and  handling  characteristics  of  the  simulation  and  the 
different  types  of  map  displays.  However,  participants  were  encouraged  to  stay  on  the  flight 
path  as  much  as  possible  because  accuracy  was  one  of  the  performance  measures  during  the 
experimental  session. 
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The  complete  two-hour  experimental  session  consisted  of  a  total  of  six  different  seventeen- 
minute  scored  scenarios,  three  of  which  were  flown  at  a  low  altitude  (200  ft)  and  three  flown  at  a 
high  altitude  (1000  ft).  These  scenarios  are  shown  graphically  in  Figure  2.2a-f,  starting  on  page 
46.  Because  terrain  was  one  of  the  hazards  employed  in  the  scenarios,  the  low  and  high  altitude 
paths  always  remained  the  same.  Care  was  taken  in  the  formulation  of  the  scenarios  to  ensure 
that  the  level  of  difficulty  was  consistent  regardless  of  the  path.  This  included  making  sure  that 
every  scenario  contained  four  hover  points,  seven  legs,  and  three  man-made  and  three  natural 
hazards  experienced  during  forward  flight  (hover  points  and  hazards  described  in  detail  later). 

The  pilots  were  told  that  they  were  carrying  a  team  of  scientists  conducting  a  geological 
survey  for  possible  oil  deposits,  which  required  a  very  precise  flight  over  the  defined  path  and 
multiple  hover  points  in  order  for  the  scientists  on  board  to  map  the  terrain  (see  Appendix  A  for 
the  complete  subject  instructions).  The  pilots  were  instructed  to  fly  the  low  altitude  flights  (200 
ft)  at  70  knots  airspeed  and  the  high  altitude  flights  (1000  ft)  at  110  knots.  This  was  done  to  try 
and  capture  characteristics  of  real  world  helicopter  flight,  in  that  pilots  usually  fly  slower  the 
lower  they  go.  They  were  also  instructed  to  fly  each  scenario  with  minimum  lateral  and  vertical 
deviations  from  the  defined  flight  path.  The  required  flight  path  was  presented  to  the  pilots  by  a 
compass  that  appeared  before  the  waypoints  and  at  the  hover  points  and  was  visible  for  20 
seconds.  The  compass  displayed  the  heading  for  the  next  leg  of  the  scenario  to  which  the  pilots 
were  supposed  to  turn.  When  the  pilots  achieved  the  new  heading,  which  was  redundantly 
displayed  by  the  on-screen  heading,  they  were  instructed  to  maintain  that  heading  until  the  next 
waypoint  or  hover  point.  If  a  participant  deviated  from  the  specified  flight  path  by  more  than 
500  feet,  a  command  arrow  appeared  on  the  map  display  to  direct  the  pilot  back  towards  the 
defined  route.  This  was  done  to  ensure  that  the  pilots  would  come  within  close  proximity  of  the 
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hazards,  which  was  needed  in  order  to  try  and  determine  the  level  of  global  situation  awareness 
attained  by  the  participants. 


The  hazards  encountered  during  the  scenarios  were  man-made  (power  lines  and  towers)  and 
natural  (terrain  and  weather).  For  the  low  altitude  scenarios,  the  power  lines  were  placed  across 
the  flight  path  at  200  ft,  whereas  they  were  placed  across  the  flight  path  at  1000  ft  for  the  high 
altitude  scenarios,  and  the  power  lines  were  attached  to  poles  positioned  1500  feet  apart.  The 
towers  were  either  200  or  1000  feet  tall,  depending  on  the  scenario,  and  were  also  placed  on  or 
near  the  flight  path.  The  weather  was  placed  beside  or  above  the  flight  path  and  reduced 
visibility  was  simulated  if  the  pilot  entered  the  weather.  The  weather  hazards  were  depicted  on 
the  map  display,  however,  they  were  not  visible  in  the  FFOV  unless  they  were  entered,  in  which 
case  FFOV  visibility  was  greatly  reduced.  The  terrain  was  visible  on  the  electronic  map  and  in 
the  forward  field  of  view  (FFOV).  All  powerlines,  weather,  and  terrain  hazards  were  accurately 
represented  on  the  map  display,  however,  the  towers  were  not  depicted  on  the  map.  This  was 
done  in  an  attempt  to  force  the  subjects  to  maintain  a  visual  scan  between  the  electronic  map 
display  and  the  FFOV. 

Each  scenario  also  contained  four  points  at  which  the  pilots  were  directed  to  hover,  which 
were  designed  to  directly  measure  the  pilot's  global  situation  awareness.  Pilots  transitioned  into 
hover  mode  by  decreasing  the  airspeed  via  the  "China  hat"  on  the  joystick,  and  were  considered 
in  hover  mode  when  their  speed  equaled  zero.  Because  of  the  need  to  ensure  that  the  pilots 
hovered  at  the  correct  spot,  an  "X"  appeared  on  the  map  display  at  the  hover  location  when  the 
pilots  were  within  1/2  mile  from  the  point.  The  "X"  disappeared  from  the  map  as  soon  as  the 
pilot  entered  the  hover  mode,  at  which  time  a  compass  with  a  predefined  direction  was  displayed 
on  the  map  for  20  seconds.  The  participants  were  instructed  to  rotate  themselves  to  point  in  the 


same  direction  as  that  indicated  on  the  compass.  After  20  seconds,  the  compass  disappeared 
from  the  map,  which  indicated  that  the  pilot  needed  to  transition  back  into  forward  flight. 
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The  scenarios  were  developed  such  that  there  were  one  of  four  possible  situations  present  at 
each  hover  point:  1)  hover  (H),  2)  hover  with  a  cross-wind  blowing  (HC),  3)  hover  with  a  hazard 
present  nearby  (HH),  and  4)  hover  with  a  hazard  present  nearby  and  a  cross-wind  blowing 
(HHC).  These  were  presented  in  a  fixed  order  for  each  scenario.  The  predefined  direction  to 
which  the  pilots  had  to  rotate  was  such  that  the  hazards  at  the  hover  points  were  never  in  the 
FFOV.  Also,  the  crosswind  began  blowing  exactly  five  seconds  after  the  pilot  entered  the  hover 
mode.  If  a  hazard  and  crosswind  were  both  present  at  a  hover  point  (HHC),  the  crosswind 
always  blew  the  aircraft  towards  the  hazard,  while  for  case  HC  where  there  was  no  hazard 
present,  the  crosswind  blew  the  aircraft  1 80  degrees  from  the  pilots  current  heading  (directly 
backwards). 

Performance  measures 

Several  performance  measures  were  collected  during  each  trial  in  an  effort  to  quantify  the 
level  of  local  guidance  and  global  situation  awareness  facilitated  by  the  different  types  of  map 
displays.  These  measures  included  flight  performance,  hazard  contacts,  direction-indicating 
tasks,  implicit  measures,  and  subjective  estimation  using  the  Situational  Awareness  Rating 
Technique  (SART). 

Flight  performance 

Lateral  and  vertical  deviations  from  the  specified  flight  path  were  measured  during  each  trial. 
The  scenarios  were  specifically  designed  to  cause  the  pilots  to  have  to  avoid  terrain  features  and 
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other  hazards,  but  they  were  instructed  to  maintain  the  defined  altitude  as  closely  as  possible, 
which  meant  that  they  were  told  to  "skim"  the  terrain  and  man-made  hazards  as  closely  as 
possible.  Therefore,  every  pilot  had  to  maneuver  to  avoid  these  hazards  (or  if  they  contacted 
them  the  measurement  is  defined  below),  but  large  increases  in  altitude  to  avoid  the  hazards 
would  result  in  a  greater  vertical  error  and  lower  performance  score.  Mean  absolute  error  (MAE) 
was  calculated  instead  of  root  mean  square  error,  in  order  to  minimize  the  potential  skewing 
effect  from  a  small  percentage  of  extreme  deviations.  On  all  trials,  vertical  and  horizontal  errors 
were  sampled  at  a  rate  of  2  Hz.  Also,  for  each  hover  point,  the  distance  between  the  defined 
hover  point  and  the  actual  hover  point  was  measured  in  feet. 

Hazard  contacts 

Each  scenario's  flight  path  was  designed  such  that  it  placed  hazards  directly  in  the  pilot's 
flight  path,  which  he  or  she  therefore  had  to  navigate  to  avoid.  The  number  of  times  a  pilot  made 
contact  with  a  hazard  and  the  type  of  hazard  involved  were  recorded. 

Direction-indicating  task 

Each  scenario  contained  randomly  positioned  pink  flags  that  were  depicted  only  on  the  IRIS 
map  display.  At  unexpected  times  during  each  trial,  all  of  the  IRIS  map  display  features, 
including  the  pink  flags,  disappeared  (i.e.,  everything  except  the  map  grid  disappeared),  the 
scenario  was  frozen,  and  the  helicopter  was  automatically  placed  in  hover  mode.  The  pilot’s  task 
was  to  use  the  joystick  to  rotate  ownship  to  point  directly  at  the  location  where  they  remembered 
the  closest  flag.  There  were  two  direction-indicating  tasks  per  trial.  One  of  these  occurred  one 
mile  before  the  subject  reached  a  flag,  and  the  other  occurred  one  half  mile  after  the  subject  had 
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passed  a  flag.  The  flags  were  always  visible  to  the  subjects  in  the  exocentric  and  2D  displays 
immediately  preceding  the  freeze,  however,  the  flag  was  never  visible  in  the  egocentric  display 
immediately  preceding  the  freeze  event  after  flag  task.  Deviations  were  recorded  as  the  angle 
difference  between  the  heading  the  pilot  rotated  to  and  the  exact  heading  of  the  closest  flag. 

Implicit  measures 

Each  trial  contained  four  defined  points  where  pilots  were  instructed  to  enter  a  hover.  One  of 
the  four  hover  points  included  a  crosswind  which  blew  the  aircraft  towards  a  hazard  (HHC).  At 
this  hover  point,  flight  control  inputs,  heading,  and  position  data  were  continuously  sampled  (at  5 
Hz)  throughout  the  duration  of  the  hover  so  that  the  appropriateness  of  control  behavior  could  be 
ascertained.  The  control  input  data  was  recorded  as  deflections  of  the  joystick  in  the  X  and  Y 
directions,  and  the  experimenters  determined  that  a  deflection  value  of  2  or  greater  indicated  a 
meaningful  movement.  When  the  control  inputs  reached  the  specified  thresholds,  the  simulation 
computer  used  the  next  five  samples  to  calculate  a  direction  vector  for  the  movement. 

Deviations  were  recorded  as  the  angle  difference  between  the  direction  of  the  crosswind  and  the 
direction  of  the  movement  vector.  Also,  the  reaction  time  from  the  onset  of  the  crosswind  until 
meaningful  compensatory  input  was  recorded. 

Subjective  estimation 

At  three  minute  intervals  throughout  each  scenario,  the  subjects  were  asked  to  verbally  rate 
his  or  her  situation  awareness  on  the  three  dimensional  SART  scale.  At  these  points  in  the 
scenario,  the  investigator  would  say  "SART"  out  loud,  which  would  cue  the  subjects  to  provide 
their  ratings.  Subjects  could  refer  to  an  index  card  attached  to  the  map  display  monitor  that 


43 


contained  the  definitions  of  the  different  dimensions  of  the  SART.  The  dimension  was  rated  on 
a  seven  point  scale  ranging  from  LOW  (1)  to  HIGH  (7). 
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Figure  2.1a:  Map  display  with  egocentric  viewpoint. 
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Figure  2.1b:  Map  display  with  exocentric  viewpoint. 
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Figure  2.1c:  Map  display  with  planar  viewpoint. 
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Figure  2.2c:  Path  3  at  200  feet 


49 


RESULTS 

All  statistical  analyses  were  performed  using  SPSS  version  8.0  for  Windows™,  and  the 
figures  were  drawn  using  Microsoft  Excel  7.0  for  Windows™.  All  error  bars  depicted  on  the 
figures  represent  a  95%  confidence  interval.  There  were  no  trial  or  path  effects. 

Flight  performance 

Horizontal 

Deviation  error  in  terms  of  horizontal  MAE  is  presented  in  Figure  3.1a  (see  page  57).  A  3x2 
repeated  measures  ANOVA  revealed  a  significant  main  effect  for  altitude  level 
[F(  1,101  )=30.4,  p<0.001],  with  greater  horizontal  errors  occurring  at  the  higher  (1000  feet) 
altitude  level.  There  was  no  significant  difference  between  the  three  display  types 
[F(2, 101)=1.3,  p=.26],  and  no  significant  interaction  was  revealed  between  display  type  and 
altitude  level  [F(2,  101)=0.2,  p=0.82]. 

Vertical 

An  analysis  similar  to  the  horizontal  error  analysis  was  performed  on  the  vertical  error  data, 
plotted  in  Figure  3.1b  (see  page  57).  Again,  the  analysis  revealed  a  significant  main  effect  for 
altitude  level  [F(l,101)=27.5,  p<0.001],  with  greater  vertical  errors  occurring  at  the  higher 
altitude  level.  No  significant  display  type  effect  [F(2, 101)=  1.9,  p=0.16]  or  display  type  by 
altitude  level  interaction  [F(2, 101)=0.52,  p=0.60]  was  revealed. 
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Stopping  distance  from  hover  points 

For  each  of  the  hover  points,  the  distance  between  the  defined  hover  point  and  the  actual  point 
where  the  subjects  stopped  to  hover  was  measured  in  feet,  as  plotted  in  Figure  3.1c  on  page  57. 
Results  from  the  3x2  repeated  measures  ANOVA  revealed  a  significant  display  type  effect 
[F(2, 426)=9.7,  p<0.001].  A  post  hoc  Tukey  test  indicated  the  2D  viewpoint  supported 
significantly  better  performance  than  the  egocentric  viewpoint  or  exocentric  viewpoint.  A 
significant  altitude  level  effect  [F(l,426)=15.3,  p<0.001],  and  a  significant  display  type  by 
altitude  level  interaction  [F(2, 426)=18.4,  p<0.001]  were  also  revealed,  the  latter  indicating  that 
only  the  exocentric  viewpoint  provided  significantly  worse  performance  at  the  higher  altitude 
level  (1000  feet)  than  at  the  lower  altitude  level  (200  feet). 

Hazard  contacts 

Powerlines 

The  probability  of  contacts  with  powerlines  was  computed  as  the  actual  number  of  contacts 
per  scenario  divided  by  the  number  of  powerlines  present  in  each  scenario.  For  the  low  altitude 
(200  feet)  level  scenarios,  there  were  three  powerlines  present,  while  for  the  high  altitude  level 
scenarios  there  were  two  powerline  hazards  present.  A  3x2  repeated  measures  ANOVA  was 
performed  on  these  contact  occurrences,  as  presented  in  Figure  3.2a  (page  58).  There  were  no 
significant  differences  revealed  for  either  the  display  type  [F(2, 102)=0.43,  p=0.65]  or  altitude 
level  [F(l,  102)=2.60,  p=0.1 1],  and  there  was  no  significant  display  type  by  altitude  interaction 
[F(2,  102)=0.90,  p=0.41]. 
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Towers 

The  probability  of  contacts  with  towers  was  computed  as  the  actual  number  of  contacts  per 
scenario  divided  by  the  number  of  towers  present  in  each  scenario.  For  the  low  altitude  (200 
feet)  level  scenarios,  there  was  one  tower  present,  while  for  the  high  altitude  level  scenarios  there 
were  two  tower  hazards  present.  A  3x2  repeated  measures  ANOVA  was  performed  on  these 
contact  occurrences,  presented  in  Figure  3.2b  on  page  58.  There  were  no  significant  differences 
revealed  for  display  type  [F(2, 102)=0.00,  p=1.00]  or  altitude  level  [F(l,  102)=1.06,  p=0.31],  and 
there  was  no  significant  display  type  by  altitude  interaction  [F(2, 102)=0.68,  p=0.51]. 

Terrain 

As  with  the  powerlines  and  towers,  the  probability  of  contacts  with  the  terrain  was  computed 
as  the  number  of  contacts  per  scenario  divided  by  the  number  of  terrain  hazards  present  in  each 
scenario.  The  low  altitude  scenarios  contained  two  terrain  hazards  while  the  high  altitude 
scenarios  only  contained  one  terrain  hazard.  It  was  possible  to  contact  a  given  terrain  feature 
more  than  once.  The  terrain  data  are  plotted  in  Figure  3.2c  (page  59).  The  3x2  repeated 
measures  ANOVA  revealed  a  display  type  effect  [F(2,  102)=5.2,  p=0.007].  A  post  hoc  Tukey 
test  indicated  that  significantly  more  terrain  collisions  occurred  with  the  use  of  the  egocentric 
viewpoint.  Although  no  significant  altitude  level  effects  [F(l,  102)=0.98,  p=0.32]  or  display 
type  by  altitude  level  interactions  [F(2, 102)= 1.92,  p=0.15]  were  revealed,  further  analysis  of  the 
egocentric  viewpoint  data  by  a  paired  samples  t-test  revealed  a  marginally  significant  difference 
between  the  altitude  levels  [t(17)=-1.84,  p=0.08],  with  more  terrain  contacts  occurring  at  the 
higher  altitude  level. 
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Weather 

For  the  lower  altitude  scenarios,  there  was  one  weather  hazard  present,  while  the  higher 
altitude  scenarios  contained  two  weather  hazards.  The  probability  of  weather  contacts  was 
computed  as  the  number  of  contacts  per  scenario  divided  by  the  number  of  weather  hazards 
present  in  each  scenario,  with  the  weather  contact  data  presented  in  Figure  3.2d  (page  59).  A 
3x2  repeated  measures  ANOVA  revealed  no  significant  display  type  [F(2,  102)=:0.80,  p=0.45]  or 
altitude  level  effect  [F(l,  102)=1.06,  p=0.31],  and  there  was  no  significant  display  type  by 
altitude  level  interaction  [F(2, 102)=0.27,  p=0.77]. 

Direction-indicating  task 

Freeze  event  before  flag 

There  was  one  freeze  event  per  scenario  where  the  freeze  occurred  one  mile  before  the  subject 
reached  the  flag.  Deviations  were  recorded  as  the  angle  difference  between  the  heading  to  which 
the  pilot  rotated  and  the  exact  heading  to  the  closest  flag.  The  angle  deviation  data  are  presented 
in  Figure  3.3a  (see  page  60),  with  smaller  values  (i.e.,  smaller  angle  differences)  indicating  better 
performance.  A  3x2  repeated  measures  ANOVA  revealed  no  significant  display  type 
[F(2, 102)=1.51,  p=0.23]  or  altitude  level  effect  [F(l,  102)=1.41,  p=0.24],  and  there  was  no 
significant  display  type  by  altitude  level  interaction  [F(2, 102)=0.30,  p=0.75]. 

Freeze  event  after flag 

There  was  one  freeze  event  per  scenario  where  the  freeze  occurred  one  half  mile  after  the 
subject  passed  the  flag.  As  with  the  freeze  event  before,  deviations  were  recorded  as  the  angle 
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difference  between  the  heading  to  which  the  pilot  rotated  and  the  exact  heading  to  the  closest 
flag,  again  with  smaller  angle  differences  indicating  better  performance.  Figure  3.3b  (page  60) 
plots  these  deviations.  A  3x2  repeated  measures  ANOVA  revealed  a  marginally  significant 
effect  for  display  type  [F(2,  102)=2.9,  p=0.06],  with  a  post  hoc  Tukey  test  revealing  that  the 
egocentric  viewpoint  provided  the  worst  performance  and  the  2D  viewpoint  supported  the  best 
performance.  No  significant  altitude  level  effect  [F(l,  102)=1.5,  p=0.22].  Also,  even  though 
Figure  3.3b  appears  to  show  an  interaction  for  the  exocentric  display,  no  significant  display  type 
by  altitude  level  interaction  [F(2, 102)=1.45,  p=0.24]  was  revealed. 

Implicit  SA  measure 

Compensatory  control  input 

One  of  the  four  hover  points  in  each  scenario  included  a  crosswind  which  blew  the  aircraft 
towards  a  hazard  (HHC).  At  this  hover  point,  deviations  were  recorded  as  the  angle  difference 
between  the  direction  of  the  compensatory  movement  vector  and  the  direction  of  the  crosswind, 
with  larger  angle  differences  indicating  better  performance,  as  seen  in  Figure  3.4a  (refer  to  page 
61).  One  of  the  18  subjects  missed  or  ignored  all  of  the  crosswinds,  therefore,  this  subject  was 
excluded  from  this  analysis.  Other  subjects  also  missed  or  did  not  provide  compensatory  control 
input  for  the  implicit  measure  at  some  of  the  hover  points.  The  frequency  with  which  this 
occurred  for  each  of  the  displays  and  altitude  levels  is  presented  in  Table  3.1. 


Egocentric 

Exocentric 

2D 

200' 

0 

1 

0 

1000’ 

3 

9 

9 

Table  3.1:  Number  of  misses  per  condition  for  which  no  compensatory  control  input  occurred 

during  the  implicit  measure. 
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A  3x2  repeated  measures  ANOVA  was  performed  on  the  angle  differences  (excluding  the 
missed  cases)  and  revealed  a  significant  main  effect  for  altitude  level  [F(l,74)=4.4,  p=0.04],  with 
better  performance  occurring  at  the  lower  altitude  level  (200  feet),  and  a  marginally  significant 
display  type  by  altitude  level  interaction  [F(2,  74)=2.36,  p=0.10],  indicating  that  only  the 
exocentric  viewpoint  provided  significantly  worse  performance  at  the  higher  altitude  level  (1000 
feet)  than  at  the  lower  altitude  level  (200  feet).  There  was  no  significant  difference  between  the 
three  display  types  [F(2,  74)=0.39,  p=0.68] 

Reaction  time 

The  reaction  time  in  seconds  from  the  onset  of  the  crosswind  at  the  HHC  hover  point  until 
meaningful  compensatory  control  input  from  the  subject  was  recorded.  If  the  subject  did  not 
respond  to  the  crosswind  (as  displayed  in  Table  3.1),  then  the  reaction  time  for  that  case  was 
excluded  from  the  analysis.  The  reaction  times  are  displayed  on  page  61  in  Figure  3.4b.  A  3x2 
repeated  measures  ANOVA  revealed  a  significant  altitude  level  effect  [F(l,  74)=7.6,  p=0.007], 
with  greater  reaction  times  occurring  at  the  higher  altitude  level.  There  was  no  significant 
difference  between  the  display  types  [F(2,  74)=0.07,  p=0.93],  and  the  display  type  by  altitude 
level  interaction  [F(2,  74)=0.96,  p=0.39]  was  also  not  significant. 

Subjective  estimation 

Throughout  each  scenario,  the  subjects  were  queried  five  different  times  as  to  their  rating  on 
each  of  the  three  SART  categories  (demand  on  attentional  resources,  supply  of  attentional 
resources  and  understanding),  with  each  category  rated  on  a  seven  point  scale  ranging  from 
LOW  (1)  to  HIGH  (7).  The  five  ratings  within  each  category  were  averaged  to  provide  on 
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overall  rating  for  each  dimension,  and  a  3x2  repeated  measures  ANOVA  was  performed  on  these 

average  ratings  for  each  of  the  three  categories.  Figures  3.5a-3.5c  (page  62)  display  the  averaged 

ratings  data.  A  significant  effect  of  altitude  was  revealed  for  demand  on  attentional  resources 

[F(l,  102)=7.3,  p=0.008]  and  supply  of  attentional  resources  [F(l,  102)=5.2,  p=. 025],  For  both 

of  these  categories,  a  higher  average  rating  was  provided  during  the  higher  altitude  scenarios.  A 

marginally  significant  altitude  effect  was  revealed  for  understanding 

[F(l,  102)=3.5,  p=0.06],  with  higher  ratings  reported  during  the  lower  altitude  scenarios.  No 

significant  display  effect  ([Fdemand(2, 102)=0.00,  p=1.00],  [F^iy^,  102)=0.09,  p=0.91],  and 

[FUnderstanding(2,  102)=1.42,  p=0.25])  or  display  type  by  altitude  level  interaction 

([Fdemand(2,  102)=0.08,  p=0.92],  [Fsupply(2, 102)=0.02,  p=0.98],  and 

[Funderstanding(2,  102)=0.79,  p=0.46])  was  revealed  for  any  of  the  three  SART  categories. 

Correlation  between  situation  awareness  measures 

As  noted  in  the  introduction,  there  is  considerable  variability  among  researchers  as  to  their 
recommendations  regarding  which  SA  measures  should  be  used.  Therefore,  we  wanted  to 
evaluate  the  degree  to  which  the  different  measures  we  used  in  this  study  agreed  or  disagreed 
with  regards  to  the  ability  of  different  display  types  and  altitudes  to  facilitate  situation  awareness. 
In  determining  the  correlation  between  the  different  measures  we  utilized  in  the  study,  we 
decided  not  to  evaluate  two  of  the  SART  categories  (demand  on  attentional  resources,  supply  of 
attentional  resources)  and  the  freeze  event  before  the  flag  task.  This  was  because  the  two 
mentioned  SART  categories  are,  in  effect,  a  measure  of  workload  and  not  S  A,  and  the  freeze 
event  before  the  flag  task  was  considered  more  of  a  perceptual  task  than  a  cognitive  task  (since 
the  flag  was  visible  in  all  three  display  viewpoints  at  the  time  of  the  freeze  event).  Therefore, 
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our  correlation  analysis  was  performed  on  what  we  felt  were  our  most  valid  measures  of  SA;  the 
implicit  performance  measure  (compensatory  control  input),  the  freeze  event  after  flag  task,  and 
the  understanding  category  of  the  SART.  The  analysis  was  done  by  correlating  each  measure's 
average  score  within  each  of  the  six  conditions  (three  display  types  by  two  altitude  levels)  with 
one  another.  Also,  high  scores  for  the  SART  and  implicit  measure  represent  better  performance, 
whereas  lower  scores  for  the  freeze  event  after  flag  measure  represent  better  performance. 

Within  each  scatterplot  representing  the  correlation  between  measures,  shown  in 
Figure  3.6a-3.6c  (refer  to  page  63-64),  each  display  type  is  depicted  by  a  different  symbol 
(egocentric=square,  exocentric=diamond,  and  2D=triangle).  Also,  those  conditions  which 
involve  high  altitude  (1000  feet)  missions  are  denoted  by  a  circle.  The  correlation  obtained 
between  the  implicit  performance  measure  and  the  freeze  event  after  flag  task  was  r=0.5 1 
(p=0.30),  between  the  implicit  performance  measure  and  SART  (understanding)  was  r=0.84 
(p=0.03),  and  between  the  freeze  event  after  flag  task  and  SART  (understanding)  was  r=0.68 
(p=0.14).  Although  only  one  of  the  correlations  was  statistically  significant  (implicit 
performance  measure  to  SART  understanding),  there  still  appears  to  be  some  degree  of 
agreement  among  the  measures,  in  that  all  correlations  were  positive,  accounting  for  at  least  50% 


of  the  shared  variance. 


Distance  (feet)  Mean  Average  Error  (feet)  Mean  Average  Error  (feet) 
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Figure  3.1a:  Lateral  tracking  error. 
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Figure  3.1b:  Vertical  tracking  error. 
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Figure  3.1c:  Stopping  distance  from  hoverpoints. 
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Figure  3.2a:  Powerline  contacts. 


Figure  3.2b:  Tower  contacts. 


Number  of  contacts  Number  of  contacts 

(proportion  per  path)  (proportion  per  path) 


59 


Figure  3.2c:  Terrain  contacts. 


Figure  3.2d:  Weather  contacts. 


Figure  3.3b:  Freeze  event  after  flag. 
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Figure  3.4a:  Compensatory  control  input  at  hoverpoint. 


Figure  3.4b:  RT  for  compensatory  control  input  at  hoverpoint. 
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Figure  3.5a:  SART-Demand  on  attentional  resources. 
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Figure  3.5b:  SART-Supply  of  attentional  resources. 
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Figure  3.5c:  SART-Understanding. 


Figure  3.6a:  Compensatory  control  input  to  freeze  after  event  correlation. 
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Figure  3.6b:  Compensatory  control  input  to  SART  (understanding)  correlation. 
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Figure  3.6c:  Freeze  after  event  to  SART  (understanding)  correlation. 
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DISCUSSION 

This  study  sought  to  examine  the  effects  that  different  frames  of  reference  for  electronic  map 
displays  have  on  pilot  performance  while  flying  multiple  rotorcraft  specific  scenarios  at  two 
different  altitudes.  The  primary  variables  of  interest  were  the  frame  of  reference  for  the  map 
display  (egocentric,  exocentric,  and  2D)  and  the  altitude  of  flight  (200  feet  and  1000  feet)  and 
their  effects  on  pilot  navigational  ability  and  global  situation  awareness.  Additionally,  the 
degree  to  which  the  situation  awareness  measures  agreed  or  disagreed  with  regards  to  the  ability 
of  the  different  displays  to  facilitate  global  situation  awareness  was  investigated.  In  the 
following  section,  we  explore  the  effects  of  the  experimental  manipulations  on  pilot  navigational 
ability  and  global  situation  awareness,  noting  in  our  examination  of  the  degree  of  agreement 
among  the  situation  awareness  measures. 

Navigational  ability 

The  present  study  did  not  find  an  advantage  for  one  type  of  map  viewpoint  over  another  in 
terms  of  vertical  or  lateral  tracking  performance  (refer  to  Figure  3.1a  and  3.1b),  which  does  not 
support  our  first  hypothesis  that  the  egocentric  display  would  provide  superior  local  guidance 
ability.  While  this  finding  is  inconsistent  with  previous  evaluations  of  egocentric  displays  by 
Barfield  et  al.  (1995),  Wickens  and  Prevett  (1995),  and  Olmos  et  al.  (1998),  it  may  be  explained 
by  the  fact  that  this  previous  research  on  immersed  displays  was  conducted  during  instrument 
flight  rules  scenarios  in  which  no  FFOV  was  presented  to  the  subjects.  These  studies  required 
subjects  to  navigate  solely  by  use  of  the  electronic  map  display,  which  meant  they  were  "eyes-in" 
throughout  the  experiment. 
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In  contrast,  this  study  incorporated  a  FFOV  that  allowed  the  subjects  to  keep  their  "eyes-out 
throughout  the  scenarios,  simulating  visual  flight  rules  (VFR)  flying.  A  similar  frame  of 
reference  study  by  Olmos  et  al.  (1997),  also  incorporating  a  FFOV,  found  results  more  consistent 
with  those  presented  here,  in  that  no  difference  in  lateral  or  vertical  tracking  performance  was 
found  between  display  viewpoints.  Olmos  et  al.  (1997)  only  evaluated  an  exocentric  and  a  2D 
coplanar  display,  however,  their  results  did  suggest  that  the  presence  of  the  FFOV  tended  to 
mute  the  differences  in  navigation  between  display  viewpoints.  Our  results  suggest  that  VFR 
flying,  in  which  a  pilot  spends  a  substantial  amount  of  time  with  their  "eyes-out",  may  diminish 
the  effect  of  display  viewpoint  in  supporting  local  guidance  to  the  point  that  all  three  of  the  map 
viewpoints  evaluated  in  this  study  (egocentric,  exocentric,  and  2D)  provide  equivalent  navigation 
performance. 

This  equivalent  performance,  however,  was  not  seen  for  two  of  the  other  flight  performance 
measures;  hazard  contacts  and  stopping  distance  from  hover  points.  The  results  revealed  a 
significant  display  effect  for  the  number  of  times  the  terrain  was  contacted  during  the  scenarios 
(refer  to  Figure  3.2c),  with  the  egocentric  display  having  a  significantly  greater  number  of 
contacts  than  the  exocentric  or  the  2D  display.  Also,  further  analysis  of  the  egocentric  viewpoint 
data  revealed  that  most  of  the  display  effect  was  due  to  more  terrain  contacts  at  the  higher 
altitude  (1000  feet).  This  result  may  be  attributed,  in  part,  to  the  egocentric  viewpoint's 
"keyhole"  view  of  the  world.  In  this  study,  the  "keyhole"  problem  seems  to  have  manifested 
itself  only  during  the  high  altitude  (1000  feet)  missions  when  the  terrain  passed  from  the  FFOV, 
but  was  still  in  front  of  the  aircraft.  Olmos  et  al.  (1998)  also  used  contacts  with  the  surrounding 
terrain  as  a  performance  measure,  and  they  too  found  that  the  egocentric  display  suffered  with 
regards  to  terrain  hazard  awareness,  because  of  problems  associated  with  estimating  the  height  of 
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approaching  hazards.  We  infer  that  this  "keyhole"  cost  for  the  egocentric  display  did  not  affect 
pilot  performance  at  the  lower  altitude  (200  feet)  level  because  of  heightened  pilot  vigilance 
concerning  potential  terrain  conflicts  associated  with  very  low  altitude  flight,  as  seen  by  the 
higher  rating  for  the  understanding  category  of  the  SART  across  all  three  displays  at  the  lower 
altitude  (refer  to  Figure  3.5c).  It  is  also  important  to  note  that  Wickens  and  Prevett  (1995) 
showed  that  attempting  to  alleviate  the  egocentric  display's  "keyhole"  view  by  expanding  the 
GFOV  results  in  distortion,  which  might  actually  make  obtaining  accurate  information  more 
difficult.  While  we  did  not  consider  the  number  of  terrain  contacts  as  one  of  our  global  situation 
awareness  measures,  because  it  only  represented  hazards  in  the  forward  flight  path,  it 
nonetheless  reveals  to  some  degree  that  the  egocentric  display  fails  to  support  hazard  awareness. 
These  drawbacks  have  important  implications  for  using  an  egocentric  display  in  a  rotorcraft 
cockpit  to  support  a  pilot's  ability  to  closely  skim  over  terrain  and  hazards. 

The  stopping  distance  from  the  hover  point  measure  also  revealed  a  significant  display  effect. 
When  the  subjects  were  required  to  transition  into  hover  mode  at  the  hover  points,  the  "X" 
depicting  the  hover  point  was  only  visible  on  the  map  display,  which  required  the  subjects  to 
obtain  all  information  concerning  the  hover  point  location  from  the  electronic  map.  Our  results 
for  this  task  were  somewhat  expected,  with  the  2D  viewpoint  providing  more  accurate 
performance  than  the  egocentric  and  exocentric  viewpoints  (refer  to  Figure  3.1c).  This  finding  is 
similar  to  those  found  by  Merwin  et  al.  (1997),  Wickens  et  al.  (1996),  and  Olmos  et  al.  (1998) 
concerning  the  ambiguity  costs  associated  with  3D  displays  when  determining  exact  locations  in 
three  dimensional  space. 

However,  an  interesting  interaction  occurred,  with  performance  at  the  higher  altitude 
markedly  worse  than  at  the  lower  altitude  only  with  the  exocentric  viewpoint.  We  infer  that  this 
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cost  at  high  altitudes  for  the  exocentric  display  may  have  resulted  from  perceptual  biases  in 
attending  to  the  drop  lines.  These  augmentations  designed  to  improve  the  altitude  coding  on  the 
exocentric  display  by  displaying  both  the  ground  shadow  and  ownship  connected  by  a  drop  line 
(refer  to  Figure  2.  lb)  have  effectively  improved  pilots'  tracking  performance,  but  these 
augmentations  have  always  been  evaluated  in  the  context  of  fixed  winged  aircraft  flying  high 
over  the  terrain  (Olmos  et  al.,  1 997;  Wickens  et  al.  1 996).  The  higher  the  altitude  of  flight,  the 
greater  the  separation,  or  distance,  between  the  representation  of  ownship  and  the  ground 
shadow.  While  these  augmentations  have  helped  resolve  the  problem  of  determining  the  pilot's 
location  in  space,  at  higher  altitudes  it  may  actually  divide  the  pilot's  attention  along  the  length  of 
the  drop  line.  This  means  that  when  a  pilot  has  to  perform  an  exact  positioning  task,  such  as 
stopping  on  the  "X"  in  this  study,  he  or  she  may  shift  attention  back  and  forth  between  the 
representation  of  ownship  and  the  ground  shadow,  which  may  have  an  adverse  effect  on 
accuracy.  In  contrast,  at  very  low  altitude  flight,  such  as  the  200  foot  level  used  in  this  study,  the 
representation  of  ownship  and  the  ground  shadow  are  practically  superimposed  on  one  another, 
which  eliminates  the  tendency  for  divided  attention  between  the  two.  Further  research  is 
warranted  to  determine  if  this  is  indeed  the  case,  which,  if  true,  might  mean  that  one  of  the  costs 
usually  associated  with  the  exocentric  display  (difficulty  in  exact  positioning  tasks  because  of 
LOS  ambiguity),  may  be  lessened  or  eliminated  for  very  low  altitude  flight. 

Thus,  the  question  of  which  electronic  map  display  viewpoint  is  most  suited  for  rotorcraft 
cockpits  may  depend  not  so  much  on  support  of  local  guidance  ability,  since  the  presence  of  the 
FFOV  seems  to  mute  the  display  differences  previously  observed  in  instrument  flight  rules 
flying,  but  rather,  the  ability  of  each  viewpoint  to  provide  sufficient  and  accurate  information  to 
support  the  unique  tasks  encountered  during  low  altitude  rotorcraft  missions.  Our  results  suggest 
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some  potentially  serious  shortcomings  for  the  egocentric  display  and  its  ability  to  provide 
adequate  information  for  terrain  avoidance.  Also,  both  3D  displays  supported  worse 
performance  overall  for  the  exact  position  task  at  the  hover  points,  but  performance  for  the 
exocentric  display  may  vary  depending  on  the  altitude  of  flight. 

Global  situation  awareness 

For  this  discussion,  we  will  focus  on  the  three  measures  which  we  decided  represented  our 
most  valid  measures  of  global  situation  awareness;  the  freeze  event  after  flag  task,  the  implicit 
performance  measure  (compensatory  control  input),  and  the  understanding  category  of  the 
SART.  The  freeze  event  after  the  flag  task  ("freeze  after”  characterizing  Endsley's  (1987) 
SAGAT  technique),  was  the  only  one  of  these  three  GSA  measures  that  revealed  a  significant 
difference  between  the  three  display  viewpoints  (refer  to  Figure  3.3b),  with  the  egocentric 
viewpoint  resulting  in  significantly  worse  performance.  This  finding  is  consistent  with  those 
found  by  Barfield  et  al.  (1995)  and  Wickens  and  Prevett  (1995),  and  may  again  be  attributed  in 
part  to  the  "keyhole"  view  of  the  world  provided  by  the  egocentric  display,  which  results  in  the 
"out  of  sight,  out  of  mind"  problem  whereby  it  is  difficult  to  maintain  awareness  of  hazards  and 
obstacles  within  the  360  degree  sphere  around  the  aircraft.  The  flags  were  never  visible  to  the 
pilots  during  the  actual  freeze  event  (since  the  IRIS  map  display  was  blanked),  however,  because 
the  exocentric  and  2D  viewpoints  provided  a  view  of  the  aircraft  within  its  surroundings,  the 
flags  were  always  visible  in  these  two  views  immediately  preceding  the  "freeze  after"  event.  On 
the  other  hand,  because  the  egocentric  viewpoint  only  provides  a  small  scale,  forward  looking 
perspective  (3D)  view  of  the  world  (i.e.,  "keyhole"  view),  the  flag  was  never  visible  in  the 


egocentric  display  immediately  preceding  the  "freeze  after"  event,  making  it  difficult  for  the 
pilots  to  remember  and  locate  the  flag's  position. 
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However,  this  measure  did  not  reveal  an  advantage  for  the  exocentric  viewpoint  over  the  2D 
viewpoint  in  terms  of  GSA,  an  equivalence  which  does  not  support  our  second  hypothesis  that 
the  exocentric  3D  display  would  facilitate  better  global  situation  awareness  than  the  2D  display. 
This  inability  to  differentiate  between  the  two  display  viewpoints  regarding  which  provides  the 
best  global  situation  awareness  is  similar  to  results  discussed  by  Rate  and  Wickens  (1993), 

Olmos  et  aL  (1998),  and  Wickens  et  al.  (1996).  One  possible  explanation  for  why  no  differences 
were  revealed  between  the  two  display  viewpoints  may  be  that  the  flags  were  depicted  on  the 
map  display  and  not  in  the  FFOV.  We  based  our  hypothesis  on  the  feet  that  the  feirly  congruent 
picture  between  the  FFOV  and  the  map  display  offered  by  the  exocentric  viewpoint  at  lower 
altitudes  (Hickox  and  Wickens,  1997),  would  outweigh  any  of  its  associated  costs.  However, 
because  the  pilots  did  not  have  to  locate  the  flags  in  the  FFOV  and  then  translate  their  location  to 
the  map,  the  benefits  from  the  congruency  between  the  FFOV  and  the  exocentric  map  display 
might  not  have  been  realized. 

Unlike  the  "freeze  after"  event,  the  implicit  performance  measure  and  the  subjective 
estimation  measure  (SART)  did  not  indicate  differences  between  the  displays.  The  implicit 
measure  required  that  the  pilots  react  to  a  crosswind  at  one  of  the  hover  points.  There  were  a 
considerable  number  of  times  this  task  was  missed  at  the  higher  altitude  (refer  to  Table  3.1), 
especially  for  the  exocentric  and  2D  display.  This  failure  may  be  attributed  to  the  amount  of 
global  optical  flow  perceived  by  the  pilots  at  the  different  altitudes  as  the  aircraft  was  perturbed 
by  turbulence  (Larish  &  Flach,  1990;  Wickens,  1992),  which  was  five  times  greater  at  the  lower 
altitude  level  than  for  the  higher  altitude  level.  This  means  that  movement  caused  by  the 
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crosswind  at  the  hover  point  was  much  more  perceptible  in  the  FFOV  at  the  200  foot  level  versus 
the  1000  foot  level,  which  would  also  explain  the  quicker  reaction  times  for  compensatory  input 
recorded  for  the  low  altitude  scenarios  versus  the  high  altitude  scenarios  (refer  to  Figure  3.4b). 

On  the  other  hand,  at  the  higher  altitude  hover  points  where  motion  from  the  crosswind  was 
much  less  noticeable  in  the  FFOV,  the  pilots  had  to  rely  more  on  the  perception  of  motion 
provided  by  the  map  display.  The  distance  a  pixel  moved  per  unit  time  across  the  screen  during 
the  crosswind  event  was  measured  for  each  display  viewpoint,  in  order  to  determine  the  saliency 
of  motion  for  each  display.  While  the  vertical  movement  per  second  was  roughly  equivalent 
between  the  egocentric,  exocentric,  and  2D  viewpoints  (approximately  0.06  cm/sec,  0.08  cm/sec, 
and  0.16  cm/sec  respectively),  the  lateral  movement  for  the  egocentric  viewpoint  was 
dramatically  greater  than  for  the  exocentric  and  2D  viewpoints  (approximately  1.95  cm/sec, 

0.127  cm/sec  and  0.16  cm/sec  respectively).  This  means  that  movement  was  much  harder  to 
detect  during  the  crosswind  event  while  using  the  exocentric  and  2D  displays,  which  would 
account  for  the  increased  number  of  misses  for  these  two  viewpoints  at  the  higher  altitude. 

Although  the  implicit  performance  measure  of  compensatory  control  input  did  not  indicate 
differences  between  the  display  viewpoints  (refer  to  Figure  3.4a),  these  data  did  reveal  a 
marginally  significant  interaction,  with  worse  performance  at  the  higher  altitude  than  at  the  lower 
altitude  only  for  the  exocentric  display.  This  interaction  followed  a  similar  pattern  to  the 
interaction  revealed  by  the  stopping  distance  from  the  hover  point  measure  (Figure  3.1c).  These 
two  tasks  were  similar  in  that  both  the  implicit  performance  task  and  the  stopping  at  the  hover 
point  task  required  the  pilots  to  stop  at  or  remain  at  an  exact  position  in  space.  We  infer  that  the 
compensatory  control  input  appropriateness  cost  for  the  exocentric  display  at  the  high  altitude 
level  may  again  be  attributable  to  the  pilots'  attention  being  divided  along  the  drop  line  between 
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the  representation  of  ownship  and  the  ground  shadow,  which  may  have  adversely  affected 
accuracy  performance.  This  finding  again  suggests  that  further  research  is  warranted  in  order  to 
determine  if  this  dividing  of  pilot  attention  along  the  drop  line  does  indeed  result  in  an  associated 
cost  in  terms  of  accuracy  for  the  exocentric  viewpoint  at  higher  altitudes. 

For  this  study,  we  also  wanted  to  evaluate  the  degree  of  association,  or  dissociation,  between 
the  different  SA  measures.  A  correlation  analysis  was  performed  between  the  implicit 
performance  measure  (compensatory  control  input),  the  "freeze  after"  event,  and  the 
understanding  category  of  the  SART  (refer  to  Figure  3.6a-3.6c).  Table  4.1  provides  a  summary 
correlation  matrix.  While  only  the  correlation  between  the  implicit  performance  measure  and  the 
SART  understanding  measure  reached  the  standard  0.05  level  of  "significance",  there  appears  to 
be  a  fairly  good  level  of  agreement  between  all  of  the  measures  in  that  all  correlations  are 
positive  and  at  least  0.50.  These  results  are  somewhat  inconsistent  with  those  found  by  Endsley 
et  al.  (1998),  in  that  those  authors  did  not  find  any  significant  correlation  between  an  objective 
measure  of  SA,  the  SAG  AT,  and  a  subjective  measure  of  SA,  the  SART.  The  relatively  high 
degree  of  agreement  between  the  SA  measures  used  in  this  study  suggests  that  the  measures  are 
in  fact  measuring  what  we  intended  them  to,  which  is  the  level  of  situation  awareness  achieved 
by  the  pilots  during  the  scenarios. 


Implicit 

Freeze  after 

SART  (understanding) 

1.00 

Freeze  after 

0.51 

1.00 

SART  (understanding) 

0.84 

0.68 

1.00 

Table  4.1:  Correlations  between  global  situation  awareness  measures. 


Despite  the  general  agreement,  however,  the  correlations  were  not  perfect.  For  example,  over 
half  of  the  variance  in  the  "freeze  after"  measure  was  not  shared  by  the  SART  understanding 
measure.  More  specifically,  the  understanding  category  of  the  SART  did  not  reveal  a  significant 
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difference  between  the  display  viewpoints  and  their  ability  to  facilitate  global  situation 
awareness  (refer  to  Figure  3.5c),  possibly  calling  into  question  that  measure's  usefulness  and 
appropriateness  for  display  design  evaluations.  In  particular,  it  failed  to  signal  the  loss  of  SA  for 
the  egocentric  viewpoint  in  terrain  contacts  (refer  to  Figure  3.2c)  and  the  difference  in  SA  within 
the  exocentric  viewpoint  in  the  implicit  measure  (compensatory  control  input)  (refer  to  Figure 
3.4a)  and  the  stopping  distance  from  the  hover  points  measure  (refer  to  Figure  3.1c).  This 
finding  of  dissociation  is  inconsistent  with  the  previous  research  by  Selcon  and  Taylor  (1990) 
concerning  the  ability  of  the  SART  to  differentiate  different  cockpit  designs,  but  it  does  support 
the  concerns  voiced  by  Endsley  (1996)  and  Endsley  et  al.  (1998)  about  possible  shortcomings  of 
the  SART.  This  insensitivity  of  the  understanding  category  of  the  SART  to  different  display 
conditions  is  particularly  troubling  considering  that  the  "freeze  after"  event,  an  objective 
measure,  did  reveal  a  significant  difference  between  the  ability  of  the  display  viewpoints  to 
support  global  situation  awareness.  This  failure  by  the  SART  to  differentiate  display  conditions 
provides  evidence  that  supports  our  third  hypothesis  that  the  SART  may  not  be  the  best  measure 
of  the  actual  level  of  GS  A  achieved  by  the  pilots. 

Hence,  the  GSA  measures  revealed  that  the  egocentric  viewpoint  suffers  in  its  ability  to 
facilitate  sufficient  global  situation  awareness.  However,  as  with  other  studies  (Rate  &  Wickens, 
1993;  Olmos  et  al.,  1998;  Wickens  et  al.,  1996),  there  remains  some  uncertainty  as  to  whether 
the  exocentric  viewpoint  or  the  2D  viewpoint  best  supports  pilot  situation  awareness. 
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CONCLUSIONS  AND  FUTURE  RESEARCH 
In  this  study,  we  attempted  to  evaluate  the  effect  different  frames  of  reference  for  electronic 
map  displays  have  on  a  pilot's  local  guidance  ability  and  global  situation  awareness  during  low 
altitude  rotorcraft  specific  missions.  Our  results  suggest  that  the  egocentric  viewpoint  should  not 
be  implemented  into  a  rotorcraft  cockpit,  because  it  does  not  provide  any  better  local  guidance 
ability  during  VFR  flying  than  the  exocentric  or  2D  viewpoints,  but  it  has  higher  costs  in  regards 
to  its  shortcomings  for  providing  sufficient  hazard  and  global  situation  awareness.  The 
exocentric  viewpoint  and  the  2D  viewpoint  both  appear  to  offer  equivalent  local  guidance  ability 
during  VFR  flying,  however,  there  is  still  uncertainty  as  to  which  of  the  two  displays  would 
provide  better  GSA  performance.  Considering  the  critical  need  for  global  situation  awareness 
during  low  altitude  rotorcraft  missions,  further  research  is  warranted  to  try  and  determine  which 
display  best  facilitates  GSA.  One  possible  area  of  interest  is  whether  the  exocentric  or  2D 
viewpoint  better  supports  the  identification  and  location  of  hazards  depicted  only  in  the  FFOV, 
which  might  reveal  the  benefits  associated  with  the  congruent  picture  offered  by  the  exocentric 
view  at  low  altitude  levels.  Also,  the  literature  reveals  several  other  variables  of  interest  relevant 
to  electronic  map  design  for  rotorcraft  applications,  such  as  scale  or  zooming,  color, 
highlighting,  and  clutter,  which  need  to  be  addressed.  Given  the  pace  of  technological 
advancements  in  aviation,  the  study  of  these  factors  is  both  timely  and  appropriate. 
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APPENDIX  A 

PLEASE  READ  CAREFULLY! 

Pilot's  Instructions 

Thank  you  for  agreeing  to  participate  in  this  research.  It  is  important  for  the  field  of  aviation 
that  new  designs  of  systems,  such  as  electronic  map  displays,  are  thoroughly  tested  before 
becoming  a  permanent  part  of  the  hardware  of  any  cockpit.  It  is  through  your  participation  that 
we  can  offer  empirical  evidence  of  the  consequences  of  implementing  such  designs. 

Mission  Scenario 


You  are  carrying  a  team  of  scientists  conducting  a  geological  survey  for  possible  oil  deposits. 

This  survey  requires  very  precise  flights  over  the  defined  paths,  in  order  to  ensure  that  the 
scientists  are  able  to  accurately  map  critical  terrain  features.  During  these  flights,  there  will  be 
several  points  where  you  are  instructed  to  hover,  so  that  aerial  photographs  can  be  taken  of  the 
surrounding  terrain.  You  will  fly  6  missions,  with  three  of  them  flown  at  an  altitude  of  200  feet 
ASL  and  a  speed  of  70  kts  and  three  flown  at  1000  feet  ASL  and  1 10  kts.  Each  mission  lasts 
approximately  17  minutes,  and  consists  of  7  legs. 

The  flying  environment  through  which  you  will  be  flying  contains  a  number  of  man-made 
hazards  (powerlines  and  towers)  and  natural  hazards  (terrain  and  weather).  It  is  of  paramount 
importance  to  maintain  the  defined  altitude  and  speed  for  each  mission,  to  ensure  accurate 
mapping  measurements.  This  means  that  you  need  to  avoid  all  hazards  encountered  during  the 
flight,  but  also  need  to  return  to  the  defined  altitude  and  path  as  soon  as  possible.  Therefore, 
vour  primary  responsibility  is  to  fly  each  mission  as  accurately  as  possible,  but  avoid  any  hazards 
that  jeopardize  the  aircraft.  All  terrain,  weather,  and  powerline  hazards  are  accurately  depicted 
on  the  electronic  map  displays,  however,  none  of  the  towers  are  depicted  on  the  maps. 

Electronic  Maos 

One  of  three  different  maps  will  be  used  for  each  of  the  6  missions;  3D  egocentric,  3D 
exocentric,  or  2D  planar.  The  3D  egocentric  map  display  presents  a  view  that  corresponds  to 
what  you  see  from  your  seat  looking  out  of  the  window.  The  3D  exocentric  display  presents  a 
view  from  the  perspective  of  being  above  and  behind  the  aircraft.  The  2D  planar  display 
presents  a  bird's  eye  view  of  the  aircraft.  See  Figure  1-3  at  the  end  of  these  instructions  for  a 
depiction  of  how  these  maps  look. 

Joystick  Handling  Characteristics 

Throughout  each  mission,  there  will  be  two  different  modes  of  flight;  forward  flight  and  hover 
mode.  For  the  experiment,  you  are  considered  in  hover  mode  only  when  the  airspeed  equals 
zero.  The  "china  hat"  on  the  joystick  controls  airspeed,  and  therefore,  it  also  controls  the 
transition  into  and  out  of  hover  mode.  The  handling  characteristics  of  the  joystick  change 
depending  on  your  phase  of  flight. 

During  forward  flight,  the  joystick  controls  the  lateral  and  vertical  axes  of  the  helicopter  in  a 
manner  consistent  with  standard  aviation  dynamics.  Pushing  forward/back  on  the  stick  causes  a 
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decrease/increase  in  pitch,  with  the  rate  of  change  of  altitude  directly  controlled  by  the  amount  of 
pitch.  Moving  the  joystick  to  the  right  or  left  makes  the  helicopter  roll  right  or  left,  respectively, 
with  the  roll  angle  directly  proportional  to  the  rate  of  change  of  heading. 

During  hover  mode,  pushing  the  joystick  in  any  direction  causes  the  helicopter  to  slew,  or 
translate,  in  that  direction,  while  maintaining  the  current  aircraft  heading.  For  example,  pushing 
the  joystick  forward  causes  the  helicopter  to  translate  (move)  forward,  but  does  not  result  in  a 
decrease  in  pitch.  Likewise,  moving  the  joystick  to  the  right  causes  the  helicopter  to  translate  to 
the  right,  but  does  not  result  in  a  rightward  roll  or  a  change  in  heading.  The  "Twist  grip"  on  the 
joystick  is  used  during  hover  mode  to  change  the  heading,  with  a  twist  to  the  left  causing  the 
aircraft  to  turn  to  the  left,  or  decrease  heading,  and  a  twist  to  the  right  causing  the  aircraft  to  turn 
to  the  right,  or  increase  heading.  Also,  the  top  and  bottom  left  side  buttons  allowed  the 
helicopter  to  ascend  or  descend  during  hover  mode.  You  will  have  an  opportunity  to  practice 
with  these  dynamics  before  the  start  of  the  experimental  trials. 

General  Procedures 

Each  mission  begins  with  the  helicopter  at  the  defined  heading,  altitude  and  speed  for  that 
particular  mission.  The  box  at  the  lower  right  comer  of  the  map  display  will  display  the  initial 
heading  to  fly,  which  you  need  to  fly  until  you  reach  a  waypoint  or  a  hover  point. 

Waypoints 

During  forward  flight,  the  need  to  turn  to  a  new  heading  is  indicated  by  the  appearance  of  a 
"compass"  on  the  left  side  of  the  screen  (refer  to  Figure  1-3).  A  fixed  red  line  on  the  compass 
depicts  the  new  heading,  and  your  current  heading  is  indicated  by  a  moving  black  line  on  the 
compass.  As  soon  as  the  compass  appears,  you  need  to  turn  to  the  new  heading,  in  order  to  align 
the  black  line  (current  heading)  with  the  red  line  (new  heading).  When  the  two  lines  are  aligned, 
the  aircraft  will  be  flying  on  the  new  heading.  This  new  heading,  which  is  redundantly  displayed 
in  the  box  at  the  lower  right  comer  of  the  map,  needs  to  be  flown  until  the  next  waypoint  or 
hover  point.  The  compass  will  be  visible  for  approximately  20  seconds. 

Hover  points 

Throughout  each  mission,  there  will  be  four  points  where  you  need  to  hover.  A  black  "X" 
indicating  the  location  of  the  hover  point  will  appear  on  the  map  display  when  you  are  within 
one-half  mile  of  the  hover  point.  You  need  to  decelerate  and  transition  into  hover  mode  such 
that  the  helicopter  is  as  close  to  the  center  of  the  "X"  as  possible,  where  the  hover  mode  is  again 
defined  as  happening  when  your  airspeed  equals  zero.  This  transition  is  performed  by  using  the 
"China  hat"  to  decrease  airspeed.  Once  your  airspeed  equals  zero  (i.e.,  you  are  in  hover  mode), 
the  "X"  disappears  from  the  map  display  and  the  compass  once  again  appears  on  the  left  side  of 
the  map.  You  need  to  use  the  twist  grip  on  the  joystick  to  rotate  the  helicopter  (current  heading 
depicted  by  the  black  line)  to  the  new  heading  (depicted  by  the  red  line).  When  the  two  lines  are 
aligned,  the  aircraft  will  be  pointed  in  the  direction  of  the  new  heading.  The  compass  will 
remain  on  the  screen  for  approximately  20  seconds.  If  a  crosswind  occurs  at  the  hover  point,  you 
need  to  try  and  maintain  the  exact  location  where  you  entered  hover  mode,  by  moving  the 
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joystick  (do  not  increase  airspeed).  As  soon  as  the  compass  disappears,  you  need  to  transition 
back  into  forward  flight  by  increasing  your  airspeed  back  to  the  defined  airspeed  for  the  mission 
(70  kts  for  200  foot  missions,  110  kts  for  1000  foot  missions),  which  is  accomplished  by  pressing 
upwards  on  the  "China  hat".  The  new  heading,  which  is  redundantly  displayed  in  the  box  at  the 
lower  right  comer  of  the  map,  needs  to  be  flown  until  the  next  waypoint  or  hover  point. 

Direction  indicating  tasks 

Within  each  mission,  there  are  randomly  positioned  pink  flags  visible  only  on  the  electronic  map 
display.  At  unexpected  times  during  each  mission,  all  features  on  the  map  display  will 
disappear,  the  scenario  will  freeze,  and  the  helicopter  will  automatically  be  placed  in  hover 
mode.  You  will  need  to  use  the  twist  grip  on  the  joystick  to  rotate  the  helicopter  to  point  directly 
at  the  closest  flag  on  the  map.  When  you  think  you  are  pointing  directly  at  the  closest  flag,  click 
the  uppermost  left  side  button.  As  soon  as  the  button  is  clicked,  the  map  display  features 
reappears  and  the  mission  resumes. 

Verbal  reports 

At  predetermined  times  throughout  each  scenario,  you  will  be  asked  to  give  a  verbal  rating  of 
your  current  state  of  awareness.  This  rating  will  be  based  on  three  different  aspects,  which  are: 

1)  Demand  on  attentional  resources— this  can  be  thought  of  as  the  instability  of  the  situation,  the 
variability  of  the  situation,  and  the  complexity  of  the  situation, 

2)  Supply  of  attentional  resources-this  can  be  thought  of  as  your  arousal  level,  your 
concentration  level,  and  how  much  your  attention  is  divided  between  tasks,  and 

3)  Understanding-this  can  be  thought  of  as  the  quantity  and  quality  of  the  information  you  are 
receiving  and  your  familiarity  with  the  situation. 

The  investigator  will  prompt  you  for  your  ratings  by  saying  "SART",  at  which  time  you  need  to 
score  each  different  aspect  on  a  scale  of  1  (Low)  to  7  (High).  A  sheet  of  paper  which  displays 
the  three  aspects  is  attached  to  the  map  display  monitor.  You  will  verbally  provide  the  number 
rating  in  the  same  order  that  the  categories  are  listed  on  the  sheet  of  paper.  For  example,  I  will 
say  "SART"  and  you  will  say  "3",  "4",  "2".  This  corresponds  to  a  rating  of  3  for  demand  on 
attentional  resources,  a  rating  of  4  for  supply  of  attentional  resources,  and  a  rating  of  2  for 
understanding.  Try  to  provide  the  ratings  quickly  so  that  you  are  not  distracted  from  your 
primary  task  of  navigation  and  hazard  avoidance. 

A  Final  Note 

Flying  the  route  as  accurately  as  possible  while  avoiding  all  obstacles  is  your  primary 
responsibility.  A  combination  of  keeping  your  eyes  "out  of  the  cockpit"  and  using  the  electronic 
maps  will  help  you  achieve  these  goals.  At  the  completion  of  the  full  experiment  (i.e.,  all 
subjects),  we  will  award  a  bonus  prize  of  $20.00  to  the  pilot  who  performs  the  best  in  terms  of 
maintaining  flight  parameters,  completing  the  missions  in  a  timely  manner,  AND  avoiding 
hazards.  Two  additional  bonus  prizes  of  $10.00  will  be  awarded  to  the  second  and  third  best 
performers.  Thank-you  again  for  your  participation. 
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Figure  1.  Egocentric  viewpoint. 
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Figure  2.  Exocentric  viewpoint 
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Figure  3.  2D  planar  viewpoint 
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